2024-09-11T07:45:23.418 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T07:45:23.433 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T07:45:23.522 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901242 branch: wip-guits-quincy-2024-09-10-2008-quincy description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} email: gabrioux@ibm.com first_in_suite: false job_id: '7901242' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-quincy-2024-09-10-2008-quincy ceph: conf: client: debug ms: 1 global: mon election default strategy: 1 ms bind msgr2: false ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) mon_bind_msgr2: false sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: cephadm-package install: ceph: flavor: default sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 extra_packages: - cephadm selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-quincy-2024-09-10-2008-quincy sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 2282 sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-quincy-2024-09-10-2008-quincy suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 targets: smithi102.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIYVZXiiaajUCxGq3TWF6Rk8Pw1maa4iVfp4OU648p7bQ3+2yOKeuhPmBE/Z5k7dVGFi5w0I2Qn4JLQIphfvOGI= smithi146.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP4e6ijC4OaveRbyfxOTzFXQACKg7Yf09FjddwGIY8fP9JKANDJeI6qLw84cfevGMDUitKhiXZrYvWza9GV741g= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install runc nvmetcli nvme-cli -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_06:53:43 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T07:45:23.522 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa; will attempt to use it 2024-09-11T07:45:23.523 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa/tasks 2024-09-11T07:45:23.523 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T07:45:23.525 INFO:teuthology.task.internal:Checking packages... 2024-09-11T07:45:23.547 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd0e6828a2016d48cf25ad84064e50742bb1c39b9' 2024-09-11T07:45:23.547 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T07:45:23.547 INFO:teuthology.packaging:ref: None 2024-09-11T07:45:23.547 INFO:teuthology.packaging:tag: None 2024-09-11T07:45:23.547 INFO:teuthology.packaging:branch: wip-guits-quincy-2024-09-10-2008-quincy 2024-09-11T07:45:23.547 INFO:teuthology.packaging:sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:45:23.548 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-quincy-2024-09-10-2008-quincy 2024-09-11T07:45:23.698 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1623.gd0e6828a 2024-09-11T07:45:23.699 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T07:45:23.706 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T07:45:23.706 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T07:45:23.723 INFO:teuthology.task.internal:Saving configuration 2024-09-11T07:45:23.735 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T07:45:23.741 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T07:45:23.767 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901242', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 07:40:19.743705', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIYVZXiiaajUCxGq3TWF6Rk8Pw1maa4iVfp4OU648p7bQ3+2yOKeuhPmBE/Z5k7dVGFi5w0I2Qn4JLQIphfvOGI='} 2024-09-11T07:45:23.789 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901242', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 07:40:19.745386', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP4e6ijC4OaveRbyfxOTzFXQACKg7Yf09FjddwGIY8fP9JKANDJeI6qLw84cfevGMDUitKhiXZrYvWza9GV741g='} 2024-09-11T07:45:23.789 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T07:45:23.795 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-11T07:45:23.795 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-11T07:45:23.795 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T07:45:23.852 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdcb8fc7ac0>, signals=[15]) 2024-09-11T07:45:23.852 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T07:45:23.858 INFO:teuthology.task.internal:Opening connections... 2024-09-11T07:45:23.858 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi102.front.sepia.ceph.com 2024-09-11T07:45:23.859 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:45:23.937 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi146.front.sepia.ceph.com 2024-09-11T07:45:23.938 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:45:24.014 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T07:45:24.021 DEBUG:teuthology.orchestra.run.smithi102:> uname -m 2024-09-11T07:45:24.039 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64 2024-09-11T07:45:24.039 DEBUG:teuthology.orchestra.run.smithi102:> cat /etc/os-release 2024-09-11T07:45:24.096 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="CentOS Stream" 2024-09-11T07:45:24.096 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="9" 2024-09-11T07:45:24.097 INFO:teuthology.orchestra.run.smithi102.stdout:ID="centos" 2024-09-11T07:45:24.097 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="rhel fedora" 2024-09-11T07:45:24.097 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="9" 2024-09-11T07:45:24.097 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el9" 2024-09-11T07:45:24.097 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T07:45:24.097 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31" 2024-09-11T07:45:24.097 INFO:teuthology.orchestra.run.smithi102.stdout:LOGO="fedora-logo-icon" 2024-09-11T07:45:24.097 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T07:45:24.097 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://centos.org/" 2024-09-11T07:45:24.097 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T07:45:24.097 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T07:45:24.097 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T07:45:24.098 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2024-09-11T07:45:24.124 DEBUG:teuthology.orchestra.run.smithi146:> uname -m 2024-09-11T07:45:24.142 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64 2024-09-11T07:45:24.142 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release 2024-09-11T07:45:24.199 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream" 2024-09-11T07:45:24.199 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="9" 2024-09-11T07:45:24.199 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos" 2024-09-11T07:45:24.200 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora" 2024-09-11T07:45:24.200 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="9" 2024-09-11T07:45:24.200 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el9" 2024-09-11T07:45:24.200 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T07:45:24.200 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31" 2024-09-11T07:45:24.200 INFO:teuthology.orchestra.run.smithi146.stdout:LOGO="fedora-logo-icon" 2024-09-11T07:45:24.200 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T07:45:24.200 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/" 2024-09-11T07:45:24.200 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T07:45:24.200 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T07:45:24.200 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T07:45:24.201 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2024-09-11T07:45:24.223 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T07:45:24.232 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T07:45:24.276 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T07:45:24.277 DEBUG:teuthology.orchestra.run.smithi102:> test '!' -e /home/ubuntu/cephtest 2024-09-11T07:45:24.279 DEBUG:teuthology.orchestra.run.smithi146:> test '!' -e /home/ubuntu/cephtest 2024-09-11T07:45:24.296 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T07:45:24.302 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T07:45:24.302 DEBUG:teuthology.orchestra.run.smithi102:> test -z $(ls -A /var/lib/ceph) 2024-09-11T07:45:24.336 DEBUG:teuthology.orchestra.run.smithi146:> test -z $(ls -A /var/lib/ceph) 2024-09-11T07:45:24.432 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T07:45:24.485 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T07:45:24.500 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-11T07:45:24.500 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-11T07:45:24.500 DEBUG:teuthology.orchestra.run.smithi102:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T07:45:24.501 DEBUG:teuthology.orchestra.run.smithi146:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T07:45:24.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:24.518 DEBUG:teuthology.orchestra.run.smithi102:> uname -r 2024-09-11T07:45:24.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:45:24.519 DEBUG:teuthology.orchestra.run.smithi146:> uname -r 2024-09-11T07:45:24.576 INFO:teuthology.orchestra.run.smithi102.stdout:5.14.0-503.el9.x86_64 2024-09-11T07:45:24.576 INFO:teuthology.task.kernel:Running kernel on smithi102: 5.14.0-503.el9.x86_64 2024-09-11T07:45:24.576 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum install -y kernel 2024-09-11T07:45:24.577 INFO:teuthology.orchestra.run.smithi146.stdout:5.14.0-503.el9.x86_64 2024-09-11T07:45:24.578 INFO:teuthology.task.kernel:Running kernel on smithi146: 5.14.0-503.el9.x86_64 2024-09-11T07:45:24.578 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum install -y kernel 2024-09-11T07:45:26.907 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-11T07:45:28.240 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.2 MB 00:01 2024-09-11T07:45:30.150 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-11T07:45:35.958 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 6.5 MB 00:02 2024-09-11T07:45:38.259 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 19 kB 00:01 2024-09-11T07:45:39.405 INFO:teuthology.orchestra.run.smithi146.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-11T07:45:44.969 INFO:teuthology.orchestra.run.smithi146.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-09-11T07:45:45.257 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 20 MB 00:17 2024-09-11T07:45:46.950 INFO:teuthology.orchestra.run.smithi146.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T07:45:46.950 INFO:teuthology.orchestra.run.smithi146.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T07:45:47.056 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:45:47.065 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:45:47.065 INFO:teuthology.orchestra.run.smithi146.stdout: Package Architecture Version Repository Size 2024-09-11T07:45:47.065 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:45:47.065 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-11T07:45:47.065 INFO:teuthology.orchestra.run.smithi146.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T07:45:47.065 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-09-11T07:45:47.065 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T07:45:47.066 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T07:45:47.066 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T07:45:47.066 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:45:47.066 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-11T07:45:47.066 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:45:47.066 INFO:teuthology.orchestra.run.smithi146.stdout:Install 4 Packages 2024-09-11T07:45:47.066 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:45:47.067 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 79 M 2024-09-11T07:45:47.067 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 126 M 2024-09-11T07:45:47.067 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-11T07:45:47.262 INFO:teuthology.orchestra.run.smithi146.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 235 kB/s | 23 kB 00:00 2024-09-11T07:45:47.646 INFO:teuthology.orchestra.run.smithi146.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-11T07:45:48.021 INFO:teuthology.orchestra.run.smithi146.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 38 MB/s | 29 MB 00:00 2024-09-11T07:45:48.338 INFO:teuthology.orchestra.run.smithi146.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 29 MB/s | 35 MB 00:01 2024-09-11T07:45:48.338 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:45:48.338 INFO:teuthology.orchestra.run.smithi146.stdout:Total 62 MB/s | 79 MB 00:01 2024-09-11T07:45:49.018 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-11T07:45:49.366 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-11T07:45:49.366 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-11T07:45:49.608 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-11T07:45:50.326 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-11T07:45:50.327 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-11T07:45:51.033 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - Extras packages 86 kB/s | 19 kB 00:00 2024-09-11T07:45:52.204 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-11T07:45:52.769 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-11T07:45:53.471 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:45:53.563 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:45:56.365 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:45:56.499 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:45:57.651 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-11T07:45:59.683 INFO:teuthology.orchestra.run.smithi102.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T07:45:59.683 INFO:teuthology.orchestra.run.smithi102.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T07:45:59.773 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:45:59.779 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:45:59.779 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-09-11T07:45:59.779 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:45:59.779 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-11T07:45:59.779 INFO:teuthology.orchestra.run.smithi102.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T07:45:59.779 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-11T07:45:59.780 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T07:45:59.780 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T07:45:59.780 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T07:45:59.780 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:45:59.780 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-11T07:45:59.780 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:45:59.780 INFO:teuthology.orchestra.run.smithi102.stdout:Install 4 Packages 2024-09-11T07:45:59.780 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:45:59.781 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 79 M 2024-09-11T07:45:59.781 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 126 M 2024-09-11T07:45:59.781 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-11T07:45:59.978 INFO:teuthology.orchestra.run.smithi102.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 229 kB/s | 23 kB 00:00 2024-09-11T07:46:00.354 INFO:teuthology.orchestra.run.smithi102.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-11T07:46:00.796 INFO:teuthology.orchestra.run.smithi102.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 35 MB/s | 29 MB 00:00 2024-09-11T07:46:01.104 INFO:teuthology.orchestra.run.smithi102.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 28 MB/s | 35 MB 00:01 2024-09-11T07:46:01.105 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:46:01.105 INFO:teuthology.orchestra.run.smithi102.stdout:Total 59 MB/s | 79 MB 00:01 2024-09-11T07:46:01.332 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:46:01.366 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:46:01.783 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-11T07:46:02.145 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-11T07:46:02.145 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-11T07:46:03.106 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-11T07:46:03.107 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-11T07:46:05.538 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-11T07:46:06.204 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:46:06.248 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:46:06.347 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:46:09.182 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:46:09.310 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:46:14.144 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:46:14.210 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:46:19.034 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:46:44.299 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:46:44.339 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:46:44.708 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:46:44.708 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:46:44.708 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:46:44.708 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:46:45.575 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:46:45.575 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:46:45.576 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-11T07:46:45.576 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T07:46:45.576 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T07:46:45.576 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T07:46:45.576 INFO:teuthology.orchestra.run.smithi146.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T07:46:45.576 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:46:45.576 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:46:45.920 DEBUG:teuthology.orchestra.run.smithi146:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:46:46.001 INFO:teuthology.orchestra.run.smithi146.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:46:46.001 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:46:46.002 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T07:46:46.002 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T07:46:46.002 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T07:46:46.002 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-11T07:46:46.002 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi146.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T07:46:46.002 DEBUG:teuthology.orchestra.run.smithi146:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:46:46.061 INFO:teuthology.orchestra.run.smithi146.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:46:46.061 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:46:46.062 DEBUG:teuthology.orchestra.run.smithi146:> sudo rpm -qi grub2-tools 2024-09-11T07:46:46.094 INFO:teuthology.orchestra.run.smithi146.stdout:Name : grub2-tools 2024-09-11T07:46:46.094 INFO:teuthology.orchestra.run.smithi146.stdout:Epoch : 1 2024-09-11T07:46:46.094 INFO:teuthology.orchestra.run.smithi146.stdout:Version : 2.06 2024-09-11T07:46:46.094 INFO:teuthology.orchestra.run.smithi146.stdout:Release : 82.el9 2024-09-11T07:46:46.094 INFO:teuthology.orchestra.run.smithi146.stdout:Architecture: x86_64 2024-09-11T07:46:46.094 INFO:teuthology.orchestra.run.smithi146.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T07:46:46.094 INFO:teuthology.orchestra.run.smithi146.stdout:Group : Unspecified 2024-09-11T07:46:46.094 INFO:teuthology.orchestra.run.smithi146.stdout:Size : 8274670 2024-09-11T07:46:46.094 INFO:teuthology.orchestra.run.smithi146.stdout:License : GPLv3+ 2024-09-11T07:46:46.094 INFO:teuthology.orchestra.run.smithi146.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T07:46:46.095 INFO:teuthology.orchestra.run.smithi146.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T07:46:46.095 INFO:teuthology.orchestra.run.smithi146.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T07:46:46.095 INFO:teuthology.orchestra.run.smithi146.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T07:46:46.095 INFO:teuthology.orchestra.run.smithi146.stdout:Packager : builder@centos.org 2024-09-11T07:46:46.095 INFO:teuthology.orchestra.run.smithi146.stdout:Vendor : CentOS 2024-09-11T07:46:46.095 INFO:teuthology.orchestra.run.smithi146.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T07:46:46.095 INFO:teuthology.orchestra.run.smithi146.stdout:Summary : Support tools for GRUB. 2024-09-11T07:46:46.095 INFO:teuthology.orchestra.run.smithi146.stdout:Description : 2024-09-11T07:46:46.095 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:46:46.095 INFO:teuthology.orchestra.run.smithi146.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T07:46:46.095 INFO:teuthology.orchestra.run.smithi146.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T07:46:46.095 INFO:teuthology.orchestra.run.smithi146.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T07:46:46.095 INFO:teuthology.orchestra.run.smithi146.stdout:hardware devices. 2024-09-11T07:46:46.095 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:46:46.096 INFO:teuthology.orchestra.run.smithi146.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T07:46:46.096 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T07:46:46.097 INFO:teuthology.task.kernel:Updating grub on smithi146 to boot 5.14.0-505.el9.x86_64 2024-09-11T07:46:46.097 DEBUG:teuthology.orchestra.run.smithi146:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T07:46:46.716 INFO:teuthology.orchestra.run.smithi146.stderr:Generating grub configuration file ... 2024-09-11T07:46:48.033 INFO:teuthology.orchestra.run.smithi146.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T07:46:48.061 INFO:teuthology.orchestra.run.smithi146.stderr:done 2024-09-11T07:46:48.063 DEBUG:teuthology.orchestra.run.smithi146:> mktemp 2024-09-11T07:46:48.081 INFO:teuthology.orchestra.run.smithi146.stdout:/tmp/tmp.390mWgCzlK 2024-09-11T07:46:48.081 DEBUG:teuthology.orchestra.run.smithi146:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.390mWgCzlK 2024-09-11T07:46:48.148 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0666 /tmp/tmp.390mWgCzlK 2024-09-11T07:46:48.292 DEBUG:teuthology.orchestra.remote:smithi146:/tmp/tmp.390mWgCzlK is 6KB 2024-09-11T07:46:48.344 DEBUG:teuthology.orchestra.run.smithi146:> rm -fr /tmp/tmp.390mWgCzlK 2024-09-11T07:46:48.359 DEBUG:teuthology.orchestra.run.smithi146:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T07:46:48.425 INFO:teuthology.orchestra.run.smithi146.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T07:46:48.425 INFO:teuthology.orchestra.run.smithi146.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T07:46:48.425 INFO:teuthology.orchestra.run.smithi146.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T07:46:48.425 INFO:teuthology.orchestra.run.smithi146.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T07:46:48.427 DEBUG:teuthology.orchestra.run.smithi146:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T07:46:48.582 DEBUG:teuthology.orchestra.run.smithi146:> sudo shutdown -r now 2024-09-11T07:46:57.424 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:46:57.465 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:46:57.813 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:46:57.813 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:46:57.813 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:46:57.813 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:46:58.661 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:46:58.662 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:46:58.662 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-11T07:46:58.662 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T07:46:58.662 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T07:46:58.662 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T07:46:58.662 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T07:46:58.662 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:46:58.662 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:46:59.022 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:46:59.104 INFO:teuthology.orchestra.run.smithi102.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:46:59.104 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:46:59.104 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T07:46:59.105 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T07:46:59.105 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T07:46:59.105 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-11T07:46:59.105 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi102.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T07:46:59.105 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:46:59.164 INFO:teuthology.orchestra.run.smithi102.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:46:59.164 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:46:59.165 DEBUG:teuthology.orchestra.run.smithi102:> sudo rpm -qi grub2-tools 2024-09-11T07:46:59.196 INFO:teuthology.orchestra.run.smithi102.stdout:Name : grub2-tools 2024-09-11T07:46:59.196 INFO:teuthology.orchestra.run.smithi102.stdout:Epoch : 1 2024-09-11T07:46:59.196 INFO:teuthology.orchestra.run.smithi102.stdout:Version : 2.06 2024-09-11T07:46:59.197 INFO:teuthology.orchestra.run.smithi102.stdout:Release : 82.el9 2024-09-11T07:46:59.197 INFO:teuthology.orchestra.run.smithi102.stdout:Architecture: x86_64 2024-09-11T07:46:59.197 INFO:teuthology.orchestra.run.smithi102.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T07:46:59.197 INFO:teuthology.orchestra.run.smithi102.stdout:Group : Unspecified 2024-09-11T07:46:59.197 INFO:teuthology.orchestra.run.smithi102.stdout:Size : 8274670 2024-09-11T07:46:59.197 INFO:teuthology.orchestra.run.smithi102.stdout:License : GPLv3+ 2024-09-11T07:46:59.197 INFO:teuthology.orchestra.run.smithi102.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T07:46:59.198 INFO:teuthology.orchestra.run.smithi102.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T07:46:59.198 INFO:teuthology.orchestra.run.smithi102.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T07:46:59.198 INFO:teuthology.orchestra.run.smithi102.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T07:46:59.198 INFO:teuthology.orchestra.run.smithi102.stdout:Packager : builder@centos.org 2024-09-11T07:46:59.198 INFO:teuthology.orchestra.run.smithi102.stdout:Vendor : CentOS 2024-09-11T07:46:59.198 INFO:teuthology.orchestra.run.smithi102.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T07:46:59.198 INFO:teuthology.orchestra.run.smithi102.stdout:Summary : Support tools for GRUB. 2024-09-11T07:46:59.198 INFO:teuthology.orchestra.run.smithi102.stdout:Description : 2024-09-11T07:46:59.198 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:46:59.200 INFO:teuthology.orchestra.run.smithi102.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T07:46:59.200 INFO:teuthology.orchestra.run.smithi102.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T07:46:59.200 INFO:teuthology.orchestra.run.smithi102.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T07:46:59.200 INFO:teuthology.orchestra.run.smithi102.stdout:hardware devices. 2024-09-11T07:46:59.200 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:46:59.200 INFO:teuthology.orchestra.run.smithi102.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T07:46:59.201 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T07:46:59.201 INFO:teuthology.task.kernel:Updating grub on smithi102 to boot 5.14.0-505.el9.x86_64 2024-09-11T07:46:59.201 DEBUG:teuthology.orchestra.run.smithi102:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T07:46:59.886 INFO:teuthology.orchestra.run.smithi102.stderr:Generating grub configuration file ... 2024-09-11T07:47:01.190 INFO:teuthology.orchestra.run.smithi102.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T07:47:01.217 INFO:teuthology.orchestra.run.smithi102.stderr:done 2024-09-11T07:47:01.220 DEBUG:teuthology.orchestra.run.smithi102:> mktemp 2024-09-11T07:47:01.236 INFO:teuthology.orchestra.run.smithi102.stdout:/tmp/tmp.fDrg2Qk62o 2024-09-11T07:47:01.236 DEBUG:teuthology.orchestra.run.smithi102:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.fDrg2Qk62o 2024-09-11T07:47:01.302 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0666 /tmp/tmp.fDrg2Qk62o 2024-09-11T07:47:01.440 DEBUG:teuthology.orchestra.remote:smithi102:/tmp/tmp.fDrg2Qk62o is 6KB 2024-09-11T07:47:01.489 DEBUG:teuthology.orchestra.run.smithi102:> rm -fr /tmp/tmp.fDrg2Qk62o 2024-09-11T07:47:01.504 DEBUG:teuthology.orchestra.run.smithi102:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T07:47:01.567 INFO:teuthology.orchestra.run.smithi102.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T07:47:01.567 INFO:teuthology.orchestra.run.smithi102.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T07:47:01.567 INFO:teuthology.orchestra.run.smithi102.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T07:47:01.567 INFO:teuthology.orchestra.run.smithi102.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T07:47:01.569 DEBUG:teuthology.orchestra.run.smithi102:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T07:47:01.712 DEBUG:teuthology.orchestra.run.smithi102:> sudo shutdown -r now 2024-09-11T07:47:18.601 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-11T07:47:18.601 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-11T07:47:18.602 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:47:31.725 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-11T07:47:31.725 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-11T07:47:31.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:47:37.153 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.146 2024-09-11T07:47:46.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-11T07:47:46.161 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:47:49.213 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.146 2024-09-11T07:47:50.209 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.102 2024-09-11T07:47:59.219 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-11T07:47:59.219 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:48:01.216 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-11T07:48:01.217 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:48:02.305 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.102 2024-09-11T07:48:14.311 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-11T07:48:14.312 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:48:19.645 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.146 2024-09-11T07:48:32.705 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.102 2024-09-11T07:48:34.647 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-11T07:48:34.648 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:48:35.058 DEBUG:teuthology.orchestra.run.smithi146:> true 2024-09-11T07:48:35.451 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-11T07:48:35.452 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-11T07:48:35.452 DEBUG:teuthology.orchestra.run.smithi146:> uname -r 2024-09-11T07:48:35.466 INFO:teuthology.orchestra.run.smithi146.stdout:5.14.0-505.el9.x86_64 2024-09-11T07:48:35.467 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T07:48:35.467 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T07:48:35.467 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T07:48:36.468 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-11T07:48:36.468 DEBUG:teuthology.orchestra.run.smithi146:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T07:48:36.576 INFO:teuthology.orchestra.run.smithi146.stdout:ttyS1 2024-09-11T07:48:36.607 DEBUG:teuthology.parallel:result is None 2024-09-11T07:48:47.706 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-11T07:48:47.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:48:48.155 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-09-11T07:48:48.522 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-11T07:48:48.522 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-11T07:48:48.522 DEBUG:teuthology.orchestra.run.smithi102:> uname -r 2024-09-11T07:48:48.536 INFO:teuthology.orchestra.run.smithi102.stdout:5.14.0-505.el9.x86_64 2024-09-11T07:48:48.537 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T07:48:48.537 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T07:48:48.537 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T07:48:49.538 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-11T07:48:49.538 DEBUG:teuthology.orchestra.run.smithi102:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T07:48:49.656 INFO:teuthology.orchestra.run.smithi102.stdout:ttyS1 2024-09-11T07:48:49.678 DEBUG:teuthology.parallel:result is None 2024-09-11T07:48:49.678 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T07:48:49.689 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T07:48:49.689 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T07:48:49.691 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T07:48:49.710 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T07:48:49.731 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T07:48:49.756 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T07:48:49.756 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T07:48:49.758 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T07:48:49.882 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T07:48:49.889 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T07:48:49.890 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T07:48:49.892 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T07:48:49.921 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:48:49.923 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:48:49.932 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:48:49.935 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:48:49.937 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T07:48:49.944 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T07:48:49.944 DEBUG:teuthology.orchestra.run.smithi102:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T07:48:49.974 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T07:48:50.005 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T07:48:50.015 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T07:48:50.015 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T07:48:50.039 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T07:48:50.062 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:48:50.133 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:48:50.170 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:48:50.250 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:48:50.289 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-11T07:48:50.289 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T07:48:50.351 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:48:50.397 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:48:50.447 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:48:50.532 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:48:50.573 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-11T07:48:50.573 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T07:48:50.640 DEBUG:teuthology.orchestra.run.smithi102:> sudo service rsyslog restart 2024-09-11T07:48:50.642 DEBUG:teuthology.orchestra.run.smithi146:> sudo service rsyslog restart 2024-09-11T07:48:50.685 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T07:48:50.733 INFO:teuthology.orchestra.run.smithi146.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T07:48:51.246 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T07:48:51.254 INFO:teuthology.task.internal:Starting timer... 2024-09-11T07:48:51.254 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T07:48:51.325 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T07:48:51.377 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-11T07:48:51.378 DEBUG:teuthology.orchestra.run.smithi102:> sudo service auditd rotate 2024-09-11T07:48:51.459 INFO:teuthology.orchestra.run.smithi102.stdout:Rotating logs: 2024-09-11T07:48:51.461 DEBUG:teuthology.orchestra.run.smithi146:> sudo service auditd rotate 2024-09-11T07:48:51.521 INFO:teuthology.orchestra.run.smithi146.stdout:Rotating logs: 2024-09-11T07:48:51.523 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T07:48:51.523 DEBUG:teuthology.orchestra.run.smithi102:> /usr/sbin/getenforce 2024-09-11T07:48:51.544 INFO:teuthology.orchestra.run.smithi102.stdout:Permissive 2024-09-11T07:48:51.544 DEBUG:teuthology.orchestra.run.smithi146:> /usr/sbin/getenforce 2024-09-11T07:48:51.576 INFO:teuthology.orchestra.run.smithi146.stdout:Permissive 2024-09-11T07:48:51.576 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi102.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi146.front.sepia.ceph.com': 'permissive'} 2024-09-11T07:48:51.577 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T07:48:51.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:51.609 DEBUG:teuthology.orchestra.run.smithi146:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T07:48:51.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:48:51.644 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T07:48:51.644 DEBUG:teuthology.orchestra.run.smithi102:> sudo /usr/sbin/setenforce permissive 2024-09-11T07:48:51.685 DEBUG:teuthology.orchestra.run.smithi146:> sudo /usr/sbin/setenforce permissive 2024-09-11T07:48:51.721 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T07:48:51.731 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-11T07:48:51.733 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-11T07:48:51.857 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T07:48:51.868 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T07:48:51.869 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi102.front.sepia.ceph.com,smithi146.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T07:52:37.803 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi102.front.sepia.ceph.com'), Remote(name='ubuntu@smithi146.front.sepia.ceph.com')] 2024-09-11T07:52:37.805 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-11T07:52:37.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:52:37.888 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-09-11T07:52:37.965 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-11T07:52:37.965 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-11T07:52:37.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:52:38.040 DEBUG:teuthology.orchestra.run.smithi146:> true 2024-09-11T07:52:38.116 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-11T07:52:38.116 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T07:52:38.127 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T07:52:38.127 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T07:52:38.128 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T07:52:38.130 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T07:52:38.130 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T07:52:38.158 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T07:52:38.173 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T07:52:38.186 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T07:52:38.198 INFO:teuthology.orchestra.run.smithi102.stderr:sudo: ntpd: command not found 2024-09-11T07:52:38.200 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T07:52:38.210 INFO:teuthology.orchestra.run.smithi102.stdout:506 Cannot talk to daemon 2024-09-11T07:52:38.224 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T07:52:38.224 INFO:teuthology.orchestra.run.smithi146.stderr:sudo: ntpd: command not found 2024-09-11T07:52:38.235 INFO:teuthology.orchestra.run.smithi146.stdout:506 Cannot talk to daemon 2024-09-11T07:52:38.238 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T07:52:38.250 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T07:52:38.264 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T07:52:38.299 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-09-11T07:52:38.302 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T07:52:38.302 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-09-11T07:52:38.303 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:52:38.303 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:52:38.303 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:52:38.303 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:52:38.325 INFO:teuthology.orchestra.run.smithi146.stderr:bash: line 1: ntpq: command not found 2024-09-11T07:52:38.329 INFO:teuthology.orchestra.run.smithi146.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T07:52:38.329 INFO:teuthology.orchestra.run.smithi146.stdout:=============================================================================== 2024-09-11T07:52:38.329 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:52:38.329 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:52:38.329 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:52:38.329 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:52:38.329 INFO:teuthology.run_tasks:Running task pexec... 2024-09-11T07:52:38.338 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-11T07:52:38.339 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi102.front.sepia.ceph.com 2024-09-11T07:52:38.339 DEBUG:teuthology.orchestra.run.smithi102:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-11T07:52:38.339 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi146.front.sepia.ceph.com 2024-09-11T07:52:38.339 DEBUG:teuthology.orchestra.run.smithi146:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-11T07:52:38.692 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:52:38.702 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:52:38.702 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-09-11T07:52:38.702 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:52:38.702 INFO:teuthology.orchestra.run.smithi102.stdout:Removing: 2024-09-11T07:52:38.702 INFO:teuthology.orchestra.run.smithi102.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-11T07:52:38.702 INFO:teuthology.orchestra.run.smithi102.stdout:Removing unused dependencies: 2024-09-11T07:52:38.702 INFO:teuthology.orchestra.run.smithi102.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-11T07:52:38.702 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:52:38.702 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-11T07:52:38.703 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:52:38.703 INFO:teuthology.orchestra.run.smithi102.stdout:Remove 2 Packages 2024-09-11T07:52:38.703 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:52:38.703 INFO:teuthology.orchestra.run.smithi102.stdout:Freed space: 5.5 M 2024-09-11T07:52:38.703 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-11T07:52:38.707 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-11T07:52:38.707 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-11T07:52:38.720 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:52:38.733 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:52:38.733 INFO:teuthology.orchestra.run.smithi146.stdout: Package Architecture Version Repository Size 2024-09-11T07:52:38.733 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:52:38.733 INFO:teuthology.orchestra.run.smithi146.stdout:Removing: 2024-09-11T07:52:38.733 INFO:teuthology.orchestra.run.smithi146.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-11T07:52:38.733 INFO:teuthology.orchestra.run.smithi146.stdout:Removing unused dependencies: 2024-09-11T07:52:38.733 INFO:teuthology.orchestra.run.smithi146.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-11T07:52:38.733 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:52:38.733 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-11T07:52:38.734 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:52:38.734 INFO:teuthology.orchestra.run.smithi146.stdout:Remove 2 Packages 2024-09-11T07:52:38.734 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:52:38.734 INFO:teuthology.orchestra.run.smithi146.stdout:Freed space: 5.5 M 2024-09-11T07:52:38.734 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-11T07:52:38.737 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-11T07:52:38.737 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-11T07:52:38.773 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-11T07:52:38.773 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-11T07:52:38.793 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-11T07:52:38.793 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-11T07:52:38.905 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-11T07:52:38.933 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-11T07:52:38.979 INFO:teuthology.orchestra.run.smithi102.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-11T07:52:39.008 INFO:teuthology.orchestra.run.smithi146.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-11T07:52:39.033 INFO:teuthology.orchestra.run.smithi102.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:52:39.069 INFO:teuthology.orchestra.run.smithi146.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:52:39.589 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:52:39.589 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-11T07:52:39.596 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:52:39.596 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-11T07:52:39.847 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-11T07:52:39.847 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:52:39.847 INFO:teuthology.orchestra.run.smithi102.stdout:Removed: 2024-09-11T07:52:39.847 INFO:teuthology.orchestra.run.smithi102.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:52:39.847 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:52:39.847 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:52:39.904 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-11T07:52:39.904 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:52:39.905 INFO:teuthology.orchestra.run.smithi146.stdout:Removed: 2024-09-11T07:52:39.905 INFO:teuthology.orchestra.run.smithi146.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:52:39.905 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:52:39.905 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:52:40.709 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:54 ago on Wed 11 Sep 2024 07:51:46 AM UTC. 2024-09-11T07:52:40.721 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:00:45 ago on Wed 11 Sep 2024 07:51:55 AM UTC. 2024-09-11T07:52:40.842 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:52:40.843 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:52:40.843 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-11T07:52:40.843 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:52:40.843 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-11T07:52:40.843 INFO:teuthology.orchestra.run.smithi102.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-11T07:52:40.843 INFO:teuthology.orchestra.run.smithi102.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-11T07:52:40.843 INFO:teuthology.orchestra.run.smithi102.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-11T07:52:40.843 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-11T07:52:40.843 INFO:teuthology.orchestra.run.smithi102.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-11T07:52:40.843 INFO:teuthology.orchestra.run.smithi102.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-11T07:52:40.843 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-11T07:52:40.844 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-11T07:52:40.844 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:52:40.844 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-11T07:52:40.844 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:52:40.844 INFO:teuthology.orchestra.run.smithi102.stdout:Install 7 Packages 2024-09-11T07:52:40.844 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:52:40.854 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 5.2 M 2024-09-11T07:52:40.854 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 19 M 2024-09-11T07:52:40.854 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-11T07:52:40.857 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:52:40.857 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:52:40.858 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-09-11T07:52:40.858 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:52:40.858 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-11T07:52:40.858 INFO:teuthology.orchestra.run.smithi146.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-11T07:52:40.858 INFO:teuthology.orchestra.run.smithi146.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-11T07:52:40.858 INFO:teuthology.orchestra.run.smithi146.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-11T07:52:40.858 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-09-11T07:52:40.858 INFO:teuthology.orchestra.run.smithi146.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-11T07:52:40.858 INFO:teuthology.orchestra.run.smithi146.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-11T07:52:40.858 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-11T07:52:40.858 INFO:teuthology.orchestra.run.smithi146.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-11T07:52:40.858 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:52:40.858 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-11T07:52:40.858 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:52:40.859 INFO:teuthology.orchestra.run.smithi146.stdout:Install 7 Packages 2024-09-11T07:52:40.859 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:52:40.886 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 5.2 M 2024-09-11T07:52:40.886 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 19 M 2024-09-11T07:52:40.886 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-11T07:52:41.074 INFO:teuthology.orchestra.run.smithi146.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 472 kB/s | 44 kB 00:00 2024-09-11T07:52:41.099 INFO:teuthology.orchestra.run.smithi146.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 886 kB/s | 106 kB 00:00 2024-09-11T07:52:41.133 INFO:teuthology.orchestra.run.smithi146.stdout:(3/7): python3-configshell-1.1.30-1.el9.noarch. 1.2 MB/s | 72 kB 00:00 2024-09-11T07:52:41.158 INFO:teuthology.orchestra.run.smithi146.stdout:(4/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.4 MB/s | 84 kB 00:00 2024-09-11T07:52:41.200 INFO:teuthology.orchestra.run.smithi146.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 4.0 MB/s | 906 kB 00:00 2024-09-11T07:52:41.250 INFO:teuthology.orchestra.run.smithi146.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 7.0 MB/s | 837 kB 00:00 2024-09-11T07:52:41.492 INFO:teuthology.orchestra.run.smithi102.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 436 kB/s | 44 kB 00:00 2024-09-11T07:52:41.517 INFO:teuthology.orchestra.run.smithi102.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 833 kB/s | 106 kB 00:00 2024-09-11T07:52:41.551 INFO:teuthology.orchestra.run.smithi102.stdout:(3/7): python3-configshell-1.1.30-1.el9.noarch. 1.2 MB/s | 72 kB 00:00 2024-09-11T07:52:41.576 INFO:teuthology.orchestra.run.smithi102.stdout:(4/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.4 MB/s | 84 kB 00:00 2024-09-11T07:52:41.609 INFO:teuthology.orchestra.run.smithi146.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 7.0 MB/s | 3.2 MB 00:00 2024-09-11T07:52:41.609 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:52:41.609 INFO:teuthology.orchestra.run.smithi146.stdout:Total 7.1 MB/s | 5.2 MB 00:00 2024-09-11T07:52:41.610 INFO:teuthology.orchestra.run.smithi102.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 4.0 MB/s | 906 kB 00:00 2024-09-11T07:52:41.660 INFO:teuthology.orchestra.run.smithi102.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 7.5 MB/s | 837 kB 00:00 2024-09-11T07:52:41.719 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-11T07:52:41.734 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-11T07:52:41.734 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-11T07:52:41.744 INFO:teuthology.orchestra.run.smithi102.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 19 MB/s | 3.2 MB 00:00 2024-09-11T07:52:41.744 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:52:41.744 INFO:teuthology.orchestra.run.smithi102.stdout:Total 5.8 MB/s | 5.2 MB 00:00 2024-09-11T07:52:41.861 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-11T07:52:41.876 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-11T07:52:41.877 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-11T07:52:42.084 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-11T07:52:42.084 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-11T07:52:42.198 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-11T07:52:42.198 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-11T07:52:42.559 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-11T07:52:42.644 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-11T07:52:42.674 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-11T07:52:42.687 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-11T07:52:42.716 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-11T07:52:42.763 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-11T07:52:42.804 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-11T07:52:42.834 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-11T07:52:42.922 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-11T07:52:42.955 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-11T07:52:43.047 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-11T07:52:43.082 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-11T07:52:43.404 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-11T07:52:43.433 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-11T07:52:43.525 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-11T07:52:43.547 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-11T07:52:43.571 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-11T07:52:43.612 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T07:52:43.677 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-11T07:52:43.739 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T07:52:44.637 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T07:52:44.637 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-11T07:52:44.637 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-11T07:52:44.638 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-11T07:52:44.638 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-11T07:52:44.638 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-11T07:52:44.638 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-11T07:52:44.685 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T07:52:44.685 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-11T07:52:44.686 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-11T07:52:44.686 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-11T07:52:44.686 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-11T07:52:44.686 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-11T07:52:44.686 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-11T07:52:45.003 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T07:52:45.003 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:52:45.003 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-11T07:52:45.003 INFO:teuthology.orchestra.run.smithi146.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:52:45.003 INFO:teuthology.orchestra.run.smithi146.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-11T07:52:45.003 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-11T07:52:45.003 INFO:teuthology.orchestra.run.smithi146.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-11T07:52:45.004 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:52:45.004 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:52:45.105 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-11T07:52:45.105 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:52:45.105 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-11T07:52:45.105 INFO:teuthology.orchestra.run.smithi102.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:52:45.105 INFO:teuthology.orchestra.run.smithi102.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-11T07:52:45.106 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-11T07:52:45.106 INFO:teuthology.orchestra.run.smithi102.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-11T07:52:45.106 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:52:45.106 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:52:45.360 DEBUG:teuthology.parallel:result is None 2024-09-11T07:52:45.506 DEBUG:teuthology.parallel:result is None 2024-09-11T07:52:45.507 INFO:teuthology.run_tasks:Running task install... 2024-09-11T07:52:45.517 DEBUG:teuthology.task.install:project ceph 2024-09-11T07:52:45.517 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'}, 'extra_packages': ['cephadm']} 2024-09-11T07:52:45.517 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'} 2024-09-11T07:52:45.517 INFO:teuthology.task.install:Using flavor: default 2024-09-11T07:52:45.522 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T07:52:45.523 INFO:teuthology.task.install:extra packages: [] 2024-09-11T07:52:45.523 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'tag': None, 'wait_for_package': False} 2024-09-11T07:52:45.524 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:52:45.526 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'tag': None, 'wait_for_package': False} 2024-09-11T07:52:45.526 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:52:45.678 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/ 2024-09-11T07:52:45.678 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1623.gd0e6828a 2024-09-11T07:52:45.682 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/ 2024-09-11T07:52:45.682 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1623.gd0e6828a 2024-09-11T07:52:45.793 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T07:52:45.793 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-11T07:52:45.793 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T07:52:45.798 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T07:52:45.798 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-11T07:52:45.798 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T07:52:45.821 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-11T07:52:45.821 DEBUG:teuthology.orchestra.run.smithi102:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d0e6828a2016d48cf25ad84064e50742bb1c39b9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T07:52:45.833 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-11T07:52:45.833 DEBUG:teuthology.orchestra.run.smithi146:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d0e6828a2016d48cf25ad84064e50742bb1c39b9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T07:52:45.888 DEBUG:teuthology.orchestra.run.smithi102:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T07:52:45.899 DEBUG:teuthology.orchestra.run.smithi146:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T07:52:45.975 DEBUG:teuthology.orchestra.run.smithi146:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T07:52:45.997 DEBUG:teuthology.orchestra.run.smithi102:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T07:52:46.021 INFO:teuthology.orchestra.run.smithi102.stdout:check_obsoletes = 1 2024-09-11T07:52:46.023 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum clean all 2024-09-11T07:52:46.039 INFO:teuthology.orchestra.run.smithi146.stdout:check_obsoletes = 1 2024-09-11T07:52:46.041 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum clean all 2024-09-11T07:52:46.262 INFO:teuthology.orchestra.run.smithi102.stdout:45 files removed 2024-09-11T07:52:46.283 INFO:teuthology.orchestra.run.smithi146.stdout:45 files removed 2024-09-11T07:52:46.290 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-radosgw 2024-09-11T07:52:46.310 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-radosgw 2024-09-11T07:52:46.834 INFO:teuthology.orchestra.run.smithi102.stdout:ceph packages for x86_64 300 kB/s | 77 kB 00:00 2024-09-11T07:52:46.866 INFO:teuthology.orchestra.run.smithi146.stdout:ceph packages for x86_64 285 kB/s | 77 kB 00:00 2024-09-11T07:52:47.110 INFO:teuthology.orchestra.run.smithi102.stdout:ceph noarch packages 47 kB/s | 11 kB 00:00 2024-09-11T07:52:47.150 INFO:teuthology.orchestra.run.smithi146.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-09-11T07:52:47.392 INFO:teuthology.orchestra.run.smithi102.stdout:ceph source packages 7.3 kB/s | 1.8 kB 00:00 2024-09-11T07:52:47.434 INFO:teuthology.orchestra.run.smithi146.stdout:ceph source packages 7.3 kB/s | 1.8 kB 00:00 2024-09-11T07:52:47.959 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-11T07:52:48.177 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-11T07:52:49.752 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-11T07:52:50.428 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - AppStream 18 MB/s | 20 MB 00:01 2024-09-11T07:52:54.621 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-11T07:52:56.052 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.5 MB 00:02 2024-09-11T07:52:56.405 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 19 kB 00:00 2024-09-11T07:52:57.512 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - Extras packages 77 kB/s | 19 kB 00:00 2024-09-11T07:52:57.601 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-11T07:52:58.266 INFO:teuthology.orchestra.run.smithi146.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-11T07:53:02.940 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-11T07:53:03.796 INFO:teuthology.orchestra.run.smithi146.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-11T07:53:05.316 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:53:05.318 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:53:05.318 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-11T07:53:05.318 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:53:05.318 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-11T07:53:05.318 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 14 M 2024-09-11T07:53:05.318 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrading: 2024-09-11T07:53:05.318 INFO:teuthology.orchestra.run.smithi102.stdout: librados2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.1 M 2024-09-11T07:53:05.319 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T07:53:05.319 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-11T07:53:05.319 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 5.0 M 2024-09-11T07:53:05.319 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 23 M 2024-09-11T07:53:05.319 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T07:53:05.319 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T07:53:05.319 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 636 k 2024-09-11T07:53:05.319 INFO:teuthology.orchestra.run.smithi102.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T07:53:05.319 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T07:53:05.319 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T07:53:05.319 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 457 k 2024-09-11T07:53:05.319 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T07:53:05.319 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 4.9 M 2024-09-11T07:53:05.320 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T07:53:05.320 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T07:53:05.320 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 45 k 2024-09-11T07:53:05.320 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 120 k 2024-09-11T07:53:05.320 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 156 k 2024-09-11T07:53:05.320 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T07:53:05.320 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 320 k 2024-09-11T07:53:05.320 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 294 k 2024-09-11T07:53:05.320 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 99 k 2024-09-11T07:53:05.320 INFO:teuthology.orchestra.run.smithi102.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T07:53:05.320 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:05.320 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-11T07:53:05.320 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:53:05.321 INFO:teuthology.orchestra.run.smithi102.stdout:Install 22 Packages 2024-09-11T07:53:05.321 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrade 2 Packages 2024-09-11T07:53:05.321 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:05.321 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 58 M 2024-09-11T07:53:05.321 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-11T07:53:06.183 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:53:06.185 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:53:06.185 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-09-11T07:53:06.185 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:53:06.186 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-11T07:53:06.186 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 14 M 2024-09-11T07:53:06.186 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrading: 2024-09-11T07:53:06.186 INFO:teuthology.orchestra.run.smithi146.stdout: librados2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.1 M 2024-09-11T07:53:06.186 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T07:53:06.187 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-09-11T07:53:06.187 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 5.0 M 2024-09-11T07:53:06.187 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 23 M 2024-09-11T07:53:06.187 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T07:53:06.187 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T07:53:06.187 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 636 k 2024-09-11T07:53:06.187 INFO:teuthology.orchestra.run.smithi146.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T07:53:06.187 INFO:teuthology.orchestra.run.smithi146.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T07:53:06.188 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T07:53:06.188 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 457 k 2024-09-11T07:53:06.188 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T07:53:06.188 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 4.9 M 2024-09-11T07:53:06.188 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T07:53:06.188 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T07:53:06.188 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 45 k 2024-09-11T07:53:06.188 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 120 k 2024-09-11T07:53:06.188 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 156 k 2024-09-11T07:53:06.188 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T07:53:06.188 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 320 k 2024-09-11T07:53:06.188 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 294 k 2024-09-11T07:53:06.189 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 99 k 2024-09-11T07:53:06.189 INFO:teuthology.orchestra.run.smithi146.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T07:53:06.189 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:06.189 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-11T07:53:06.189 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:53:06.189 INFO:teuthology.orchestra.run.smithi146.stdout:Install 22 Packages 2024-09-11T07:53:06.189 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrade 2 Packages 2024-09-11T07:53:06.189 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:06.190 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 58 M 2024-09-11T07:53:06.190 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-11T07:53:06.208 INFO:teuthology.orchestra.run.smithi102.stdout:(1/24): ceph-base-17.2.7-1623.gd0e6828a.el9.x86 12 MB/s | 5.0 MB 00:00 2024-09-11T07:53:06.284 INFO:teuthology.orchestra.run.smithi102.stdout:(2/24): ceph-selinux-17.2.7-1623.gd0e6828a.el9. 336 kB/s | 25 kB 00:00 2024-09-11T07:53:06.367 INFO:teuthology.orchestra.run.smithi102.stdout:(3/24): libcephfs2-17.2.7-1623.gd0e6828a.el9.x8 7.5 MB/s | 636 kB 00:00 2024-09-11T07:53:06.442 INFO:teuthology.orchestra.run.smithi102.stdout:(4/24): libradosstriper1-17.2.7-1623.gd0e6828a. 6.0 MB/s | 457 kB 00:00 2024-09-11T07:53:06.734 INFO:teuthology.orchestra.run.smithi102.stdout:(5/24): librgw2-17.2.7-1623.gd0e6828a.el9.x86_6 17 MB/s | 4.9 MB 00:00 2024-09-11T07:53:06.885 INFO:teuthology.orchestra.run.smithi102.stdout:(6/24): ceph-radosgw-17.2.7-1623.gd0e6828a.el9. 13 MB/s | 14 MB 00:01 2024-09-11T07:53:06.910 INFO:teuthology.orchestra.run.smithi102.stdout:(7/24): python3-ceph-argparse-17.2.7-1623.gd0e6 258 kB/s | 45 kB 00:00 2024-09-11T07:53:06.994 INFO:teuthology.orchestra.run.smithi102.stdout:(8/24): python3-ceph-common-17.2.7-1623.gd0e682 1.1 MB/s | 120 kB 00:00 2024-09-11T07:53:07.102 INFO:teuthology.orchestra.run.smithi102.stdout:(9/24): python3-rados-17.2.7-1623.gd0e6828a.el9 2.9 MB/s | 320 kB 00:00 2024-09-11T07:53:07.152 INFO:teuthology.orchestra.run.smithi102.stdout:(10/24): python3-rbd-17.2.7-1623.gd0e6828a.el9. 5.8 MB/s | 294 kB 00:00 2024-09-11T07:53:07.156 INFO:teuthology.orchestra.run.smithi146.stdout:(1/24): ceph-base-17.2.7-1623.gd0e6828a.el9.x86 7.9 MB/s | 5.0 MB 00:00 2024-09-11T07:53:07.220 INFO:teuthology.orchestra.run.smithi102.stdout:(11/24): python3-rgw-17.2.7-1623.gd0e6828a.el9. 1.4 MB/s | 99 kB 00:00 2024-09-11T07:53:07.223 INFO:teuthology.orchestra.run.smithi146.stdout:(2/24): ceph-selinux-17.2.7-1623.gd0e6828a.el9. 377 kB/s | 25 kB 00:00 2024-09-11T07:53:07.348 INFO:teuthology.orchestra.run.smithi146.stdout:(3/24): libcephfs2-17.2.7-1623.gd0e6828a.el9.x8 5.0 MB/s | 636 kB 00:00 2024-09-11T07:53:07.481 INFO:teuthology.orchestra.run.smithi146.stdout:(4/24): libradosstriper1-17.2.7-1623.gd0e6828a. 3.4 MB/s | 457 kB 00:00 2024-09-11T07:53:07.495 INFO:teuthology.orchestra.run.smithi102.stdout:(12/24): ceph-common-17.2.7-1623.gd0e6828a.el9. 13 MB/s | 23 MB 00:01 2024-09-11T07:53:07.520 INFO:teuthology.orchestra.run.smithi102.stdout:(13/24): python3-cephfs-17.2.7-1623.gd0e6828a.e 256 kB/s | 156 kB 00:00 2024-09-11T07:53:07.570 INFO:teuthology.orchestra.run.smithi102.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 457 kB/s | 160 kB 00:00 2024-09-11T07:53:07.604 INFO:teuthology.orchestra.run.smithi102.stdout:(15/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 417 kB/s | 45 kB 00:00 2024-09-11T07:53:07.654 INFO:teuthology.orchestra.run.smithi102.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 846 kB/s | 42 kB 00:00 2024-09-11T07:53:07.688 INFO:teuthology.orchestra.run.smithi102.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-11T07:53:07.738 INFO:teuthology.orchestra.run.smithi102.stdout:(18/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 3.6 MB/s | 309 kB 00:00 2024-09-11T07:53:07.764 INFO:teuthology.orchestra.run.smithi102.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 2.9 MB/s | 67 kB 00:00 2024-09-11T07:53:07.789 INFO:teuthology.orchestra.run.smithi102.stdout:(20/24): liboath-2.6.7-2.el9.x86_64.rpm 484 kB/s | 49 kB 00:00 2024-09-11T07:53:07.815 INFO:teuthology.orchestra.run.smithi146.stdout:(5/24): ceph-common-17.2.7-1623.gd0e6828a.el9.x 17 MB/s | 23 MB 00:01 2024-09-11T07:53:07.839 INFO:teuthology.orchestra.run.smithi102.stdout:(21/24): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-11T07:53:07.864 INFO:teuthology.orchestra.run.smithi102.stdout:(22/24): librdkafka-1.6.1-102.el9.x86_64.rpm 1.9 MB/s | 662 kB 00:00 2024-09-11T07:53:07.865 INFO:teuthology.orchestra.run.smithi146.stdout:(6/24): python3-ceph-argparse-17.2.7-1623.gd0e6 911 kB/s | 45 kB 00:00 2024-09-11T07:53:07.924 INFO:teuthology.orchestra.run.smithi146.stdout:(7/24): python3-ceph-common-17.2.7-1623.gd0e682 2.0 MB/s | 120 kB 00:00 2024-09-11T07:53:08.048 INFO:teuthology.orchestra.run.smithi102.stdout:(23/24): librbd1-17.2.7-1623.gd0e6828a.el9.x86_ 14 MB/s | 2.9 MB 00:00 2024-09-11T07:53:08.083 INFO:teuthology.orchestra.run.smithi146.stdout:(8/24): ceph-radosgw-17.2.7-1623.gd0e6828a.el9. 9.3 MB/s | 14 MB 00:01 2024-09-11T07:53:08.116 INFO:teuthology.orchestra.run.smithi146.stdout:(9/24): python3-cephfs-17.2.7-1623.gd0e6828a.el 814 kB/s | 156 kB 00:00 2024-09-11T07:53:08.191 INFO:teuthology.orchestra.run.smithi146.stdout:(10/24): librgw2-17.2.7-1623.gd0e6828a.el9.x86_ 6.9 MB/s | 4.9 MB 00:00 2024-09-11T07:53:08.225 INFO:teuthology.orchestra.run.smithi146.stdout:(11/24): python3-rados-17.2.7-1623.gd0e6828a.el 2.2 MB/s | 320 kB 00:00 2024-09-11T07:53:08.250 INFO:teuthology.orchestra.run.smithi146.stdout:(12/24): python3-rbd-17.2.7-1623.gd0e6828a.el9. 2.1 MB/s | 294 kB 00:00 2024-09-11T07:53:08.275 INFO:teuthology.orchestra.run.smithi146.stdout:(13/24): python3-rgw-17.2.7-1623.gd0e6828a.el9. 1.2 MB/s | 99 kB 00:00 2024-09-11T07:53:08.365 INFO:teuthology.orchestra.run.smithi102.stdout:(24/24): librados2-17.2.7-1623.gd0e6828a.el9.x8 5.3 MB/s | 3.1 MB 00:00 2024-09-11T07:53:08.370 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:53:08.370 INFO:teuthology.orchestra.run.smithi102.stdout:Total 19 MB/s | 58 MB 00:03 2024-09-11T07:53:08.434 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-11T07:53:08.467 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-11T07:53:08.468 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-11T07:53:08.626 INFO:teuthology.orchestra.run.smithi146.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 121 kB/s | 45 kB 00:00 2024-09-11T07:53:08.801 INFO:teuthology.orchestra.run.smithi146.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 278 kB/s | 160 kB 00:00 2024-09-11T07:53:08.910 INFO:teuthology.orchestra.run.smithi146.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 390 kB/s | 42 kB 00:00 2024-09-11T07:53:08.934 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-11T07:53:08.934 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-11T07:53:08.968 INFO:teuthology.orchestra.run.smithi146.stdout:(17/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 5.2 MB/s | 309 kB 00:00 2024-09-11T07:53:08.985 INFO:teuthology.orchestra.run.smithi146.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 2.9 MB/s | 49 kB 00:00 2024-09-11T07:53:09.010 INFO:teuthology.orchestra.run.smithi146.stdout:(19/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 760 kB/s | 292 kB 00:00 2024-09-11T07:53:09.044 INFO:teuthology.orchestra.run.smithi146.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-11T07:53:09.103 INFO:teuthology.orchestra.run.smithi146.stdout:(21/24): librdkafka-1.6.1-102.el9.x86_64.rpm 801 kB/s | 662 kB 00:00 2024-09-11T07:53:09.153 INFO:teuthology.orchestra.run.smithi146.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-11T07:53:09.303 INFO:teuthology.orchestra.run.smithi146.stdout:(23/24): librbd1-17.2.7-1623.gd0e6828a.el9.x86_ 15 MB/s | 2.9 MB 00:00 2024-09-11T07:53:09.362 INFO:teuthology.orchestra.run.smithi146.stdout:(24/24): librados2-17.2.7-1623.gd0e6828a.el9.x8 9.9 MB/s | 3.1 MB 00:00 2024-09-11T07:53:09.366 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:53:09.366 INFO:teuthology.orchestra.run.smithi146.stdout:Total 18 MB/s | 58 MB 00:03 2024-09-11T07:53:09.432 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-11T07:53:09.467 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-11T07:53:09.467 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-11T07:53:09.930 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-11T07:53:09.931 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-11T07:53:10.109 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-11T07:53:10.456 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-11T07:53:10.567 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:53:10.643 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:53:10.720 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T07:53:10.768 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:53:10.857 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:53:10.911 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-11T07:53:10.941 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-11T07:53:11.137 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-11T07:53:11.381 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-11T07:53:11.413 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T07:53:11.477 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T07:53:11.481 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-11T07:53:11.513 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-11T07:53:11.557 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 10/26 2024-09-11T07:53:11.581 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:53:11.593 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T07:53:11.655 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T07:53:11.672 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:53:11.683 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T07:53:11.748 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T07:53:11.750 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T07:53:11.790 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:53:11.802 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-11T07:53:11.833 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-11T07:53:11.865 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-11T07:53:11.879 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:53:11.932 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-11T07:53:11.962 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-11T07:53:12.168 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-11T07:53:12.220 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T07:53:12.295 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T07:53:12.378 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 19/26 2024-09-11T07:53:12.405 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-11T07:53:12.416 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 20/26 2024-09-11T07:53:12.443 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T07:53:12.506 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/26 2024-09-11T07:53:12.534 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-11T07:53:12.578 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 10/26 2024-09-11T07:53:12.614 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T07:53:12.676 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T07:53:12.704 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T07:53:12.770 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 13/26 2024-09-11T07:53:12.821 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-11T07:53:12.879 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-11T07:53:12.912 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-11T07:53:13.205 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-11T07:53:13.249 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T07:53:13.340 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/26 2024-09-11T07:53:13.413 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 19/26 2024-09-11T07:53:13.452 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 20/26 2024-09-11T07:53:14.830 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:53:14.871 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:53:15.407 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:53:15.449 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T07:53:15.545 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T07:53:15.545 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T07:53:15.545 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T07:53:15.545 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:15.592 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:53:15.939 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:53:15.981 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:53:16.498 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 21/26 2024-09-11T07:53:16.534 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T07:53:16.615 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/26 2024-09-11T07:53:16.615 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T07:53:16.615 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T07:53:16.616 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:16.669 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:53:25.507 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:53:25.507 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /sys 2024-09-11T07:53:25.508 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /proc 2024-09-11T07:53:25.508 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /mnt 2024-09-11T07:53:25.508 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /var/tmp 2024-09-11T07:53:25.508 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /home 2024-09-11T07:53:25.508 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /root 2024-09-11T07:53:25.508 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /tmp 2024-09-11T07:53:25.508 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:26.455 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:53:26.455 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /sys 2024-09-11T07:53:26.455 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /proc 2024-09-11T07:53:26.455 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /mnt 2024-09-11T07:53:26.455 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /var/tmp 2024-09-11T07:53:26.455 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /home 2024-09-11T07:53:26.455 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /root 2024-09-11T07:53:26.455 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /tmp 2024-09-11T07:53:26.455 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:26.808 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T07:53:26.893 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T07:53:26.893 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:53:26.893 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T07:53:26.893 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T07:53:26.893 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T07:53:26.893 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:26.895 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T07:53:26.986 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T07:53:26.986 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:53:27.784 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T07:53:27.863 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/26 2024-09-11T07:53:27.863 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:53:27.863 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T07:53:27.863 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T07:53:27.863 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T07:53:27.863 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:27.865 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T07:53:27.954 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-11T07:53:27.954 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:53:28.189 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:53:28.189 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/26 2024-09-11T07:53:28.189 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:53:28.189 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T07:53:28.190 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:53:28.190 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/26 2024-09-11T07:53:28.190 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 6/26 2024-09-11T07:53:28.190 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 7/26 2024-09-11T07:53:28.190 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 8/26 2024-09-11T07:53:28.190 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 9/26 2024-09-11T07:53:28.190 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 10/26 2024-09-11T07:53:28.190 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T07:53:28.190 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T07:53:28.190 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 13/26 2024-09-11T07:53:28.190 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-11T07:53:28.190 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-11T07:53:28.191 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-11T07:53:28.191 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-11T07:53:28.191 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-11T07:53:28.191 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-11T07:53:28.191 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-11T07:53:28.191 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-11T07:53:28.191 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-11T07:53:28.191 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:53:28.191 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-11T07:53:28.191 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 25/26 2024-09-11T07:53:29.192 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:53:29.192 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:29.192 INFO:teuthology.orchestra.run.smithi102.stdout:Upgraded: 2024-09-11T07:53:29.192 INFO:teuthology.orchestra.run.smithi102.stdout: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:29.193 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:29.193 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-11T07:53:29.193 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:29.193 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:29.193 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:29.193 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:29.193 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T07:53:29.193 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:29.193 INFO:teuthology.orchestra.run.smithi102.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T07:53:29.193 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T07:53:29.193 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T07:53:29.193 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:29.193 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T07:53:29.194 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:29.194 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T07:53:29.194 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T07:53:29.194 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:29.194 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:29.194 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:29.194 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T07:53:29.194 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:29.194 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:29.194 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:29.194 INFO:teuthology.orchestra.run.smithi102.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T07:53:29.194 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:29.195 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:53:29.240 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:53:29.240 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/26 2024-09-11T07:53:29.240 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/26 2024-09-11T07:53:29.240 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/26 2024-09-11T07:53:29.240 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/26 2024-09-11T07:53:29.241 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/26 2024-09-11T07:53:29.241 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_6 6/26 2024-09-11T07:53:29.241 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 7/26 2024-09-11T07:53:29.241 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9. 8/26 2024-09-11T07:53:29.241 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x8 9/26 2024-09-11T07:53:29.241 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 10/26 2024-09-11T07:53:29.241 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/26 2024-09-11T07:53:29.241 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/26 2024-09-11T07:53:29.241 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 13/26 2024-09-11T07:53:29.241 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-11T07:53:29.241 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-11T07:53:29.241 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-11T07:53:29.242 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-11T07:53:29.242 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-11T07:53:29.242 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-11T07:53:29.242 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-11T07:53:29.242 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-11T07:53:29.242 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-11T07:53:29.242 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/26 2024-09-11T07:53:29.242 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-11T07:53:29.242 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 25/26 2024-09-11T07:53:29.592 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-test 2024-09-11T07:53:30.156 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:28 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:53:30.238 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-11T07:53:30.238 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:30.238 INFO:teuthology.orchestra.run.smithi146.stdout:Upgraded: 2024-09-11T07:53:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:30.239 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-11T07:53:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T07:53:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T07:53:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T07:53:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T07:53:30.239 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T07:53:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T07:53:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T07:53:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T07:53:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:30.240 INFO:teuthology.orchestra.run.smithi146.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T07:53:30.241 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:30.241 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:53:30.292 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:53:30.292 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:53:30.292 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-11T07:53:30.292 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:53:30.292 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-11T07:53:30.293 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 48 M 2024-09-11T07:53:30.293 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-11T07:53:30.293 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 163 k 2024-09-11T07:53:30.293 INFO:teuthology.orchestra.run.smithi102.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T07:53:30.293 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T07:53:30.293 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:30.293 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-11T07:53:30.293 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:53:30.293 INFO:teuthology.orchestra.run.smithi102.stdout:Install 4 Packages 2024-09-11T07:53:30.294 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:30.294 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 48 M 2024-09-11T07:53:30.294 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 193 M 2024-09-11T07:53:30.294 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-11T07:53:30.550 INFO:teuthology.orchestra.run.smithi102.stdout:(1/4): libcephsqlite-17.2.7-1623.gd0e6828a.el9. 779 kB/s | 163 kB 00:00 2024-09-11T07:53:30.571 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-test 2024-09-11T07:53:30.575 INFO:teuthology.orchestra.run.smithi102.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.3 MB/s | 304 kB 00:00 2024-09-11T07:53:30.709 INFO:teuthology.orchestra.run.smithi102.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 401 kB/s | 64 kB 00:00 2024-09-11T07:53:31.138 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:00:28 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:53:31.270 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:53:31.271 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:53:31.271 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-09-11T07:53:31.271 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:53:31.271 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-11T07:53:31.271 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 48 M 2024-09-11T07:53:31.272 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-09-11T07:53:31.272 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 163 k 2024-09-11T07:53:31.272 INFO:teuthology.orchestra.run.smithi146.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T07:53:31.272 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T07:53:31.272 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:31.272 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-11T07:53:31.272 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:53:31.272 INFO:teuthology.orchestra.run.smithi146.stdout:Install 4 Packages 2024-09-11T07:53:31.272 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:31.273 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 48 M 2024-09-11T07:53:31.273 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 193 M 2024-09-11T07:53:31.273 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-11T07:53:31.607 INFO:teuthology.orchestra.run.smithi146.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-11T07:53:31.632 INFO:teuthology.orchestra.run.smithi146.stdout:(2/4): libcephsqlite-17.2.7-1623.gd0e6828a.el9. 678 kB/s | 163 kB 00:00 2024-09-11T07:53:31.657 INFO:teuthology.orchestra.run.smithi146.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-11T07:53:31.826 INFO:teuthology.orchestra.run.smithi102.stdout:(4/4): ceph-test-17.2.7-1623.gd0e6828a.el9.x86_ 32 MB/s | 48 MB 00:01 2024-09-11T07:53:31.828 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:53:31.828 INFO:teuthology.orchestra.run.smithi102.stdout:Total 32 MB/s | 48 MB 00:01 2024-09-11T07:53:31.856 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-11T07:53:31.876 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-11T07:53:31.876 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-11T07:53:32.243 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-11T07:53:32.243 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-11T07:53:32.777 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-11T07:53:32.856 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-11T07:53:32.883 INFO:teuthology.orchestra.run.smithi146.stdout:(4/4): ceph-test-17.2.7-1623.gd0e6828a.el9.x86_ 32 MB/s | 48 MB 00:01 2024-09-11T07:53:32.885 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:53:32.885 INFO:teuthology.orchestra.run.smithi146.stdout:Total 30 MB/s | 48 MB 00:01 2024-09-11T07:53:32.899 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-11T07:53:32.913 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-11T07:53:32.928 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T07:53:32.933 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-11T07:53:32.933 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-11T07:53:33.299 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-11T07:53:33.300 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-11T07:53:33.833 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-11T07:53:33.910 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-11T07:53:33.948 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-11T07:53:33.976 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T07:53:37.358 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T07:53:37.427 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T07:53:38.399 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/4 2024-09-11T07:53:38.474 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T07:53:39.749 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T07:53:39.749 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/4 2024-09-11T07:53:39.749 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/4 2024-09-11T07:53:39.749 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-11T07:53:40.080 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-11T07:53:40.080 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:40.080 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-11T07:53:40.081 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:40.081 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:40.081 INFO:teuthology.orchestra.run.smithi102.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T07:53:40.081 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T07:53:40.081 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:40.081 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:53:40.398 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph 2024-09-11T07:53:40.814 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/4 2024-09-11T07:53:40.814 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/4 2024-09-11T07:53:40.814 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/4 2024-09-11T07:53:40.814 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-11T07:53:40.965 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:38 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:53:41.110 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:53:41.112 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:53:41.112 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-11T07:53:41.112 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:53:41.112 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-11T07:53:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: ceph x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 6.5 k 2024-09-11T07:53:41.113 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-11T07:53:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.1 M 2024-09-11T07:53:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 1.4 M 2024-09-11T07:53:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 240 k 2024-09-11T07:53:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.9 M 2024-09-11T07:53:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 15 M 2024-09-11T07:53:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T07:53:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T07:53:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T07:53:41.113 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T07:53:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T07:53:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T07:53:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T07:53:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections 2024-09-11T07:53:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-11T07:53:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T07:53:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T07:53:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T07:53:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T07:53:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T07:53:41.114 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T07:53:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T07:53:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T07:53:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T07:53:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T07:53:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T07:53:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T07:53:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T07:53:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T07:53:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T07:53:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T07:53:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T07:53:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T07:53:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T07:53:41.115 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T07:53:41.116 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:41.116 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-11T07:53:41.116 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:53:41.116 INFO:teuthology.orchestra.run.smithi102.stdout:Install 34 Packages 2024-09-11T07:53:41.116 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:41.116 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 27 M 2024-09-11T07:53:41.116 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 96 M 2024-09-11T07:53:41.117 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-11T07:53:41.134 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-11T07:53:41.134 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:41.134 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-11T07:53:41.134 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:41.134 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:41.135 INFO:teuthology.orchestra.run.smithi146.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T07:53:41.135 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T07:53:41.135 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:41.135 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:53:41.437 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph 2024-09-11T07:53:41.706 INFO:teuthology.orchestra.run.smithi102.stdout:(1/34): ceph-17.2.7-1623.gd0e6828a.el9.x86_64.r 53 kB/s | 6.5 kB 00:00 2024-09-11T07:53:41.889 INFO:teuthology.orchestra.run.smithi102.stdout:(2/34): ceph-mgr-17.2.7-1623.gd0e6828a.el9.x86_ 4.6 MB/s | 1.4 MB 00:00 2024-09-11T07:53:41.931 INFO:teuthology.orchestra.run.smithi102.stdout:(3/34): ceph-mds-17.2.7-1623.gd0e6828a.el9.x86_ 5.9 MB/s | 2.1 MB 00:00 2024-09-11T07:53:42.003 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:00:39 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:53:42.048 INFO:teuthology.orchestra.run.smithi102.stdout:(4/34): ceph-mon-17.2.7-1623.gd0e6828a.el9.x86_ 11 MB/s | 3.9 MB 00:00 2024-09-11T07:53:42.073 INFO:teuthology.orchestra.run.smithi102.stdout:(5/34): ceph-mgr-modules-core-17.2.7-1623.gd0e6 1.7 MB/s | 240 kB 00:00 2024-09-11T07:53:42.141 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:53:42.143 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:53:42.143 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-09-11T07:53:42.143 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:53:42.143 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-11T07:53:42.143 INFO:teuthology.orchestra.run.smithi146.stdout: ceph x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 6.5 k 2024-09-11T07:53:42.143 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-09-11T07:53:42.143 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.1 M 2024-09-11T07:53:42.143 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 1.4 M 2024-09-11T07:53:42.143 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 240 k 2024-09-11T07:53:42.144 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.9 M 2024-09-11T07:53:42.144 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 15 M 2024-09-11T07:53:42.144 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T07:53:42.144 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T07:53:42.144 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T07:53:42.144 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T07:53:42.144 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T07:53:42.144 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T07:53:42.144 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T07:53:42.144 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-collections 2024-09-11T07:53:42.144 INFO:teuthology.orchestra.run.smithi146.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-11T07:53:42.144 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T07:53:42.145 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T07:53:42.145 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T07:53:42.145 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T07:53:42.145 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T07:53:42.145 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T07:53:42.145 INFO:teuthology.orchestra.run.smithi146.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T07:53:42.145 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T07:53:42.145 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T07:53:42.145 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T07:53:42.145 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T07:53:42.145 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T07:53:42.145 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T07:53:42.145 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T07:53:42.145 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T07:53:42.146 INFO:teuthology.orchestra.run.smithi146.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T07:53:42.146 INFO:teuthology.orchestra.run.smithi146.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T07:53:42.146 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T07:53:42.146 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T07:53:42.146 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T07:53:42.146 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:42.146 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-11T07:53:42.146 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:53:42.146 INFO:teuthology.orchestra.run.smithi146.stdout:Install 34 Packages 2024-09-11T07:53:42.146 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:42.147 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 27 M 2024-09-11T07:53:42.147 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 96 M 2024-09-11T07:53:42.147 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-11T07:53:42.207 INFO:teuthology.orchestra.run.smithi102.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-09-11T07:53:42.249 INFO:teuthology.orchestra.run.smithi102.stdout:(7/34): python3-cryptography-36.0.1-4.el9.x86_6 7.0 MB/s | 1.2 MB 00:00 2024-09-11T07:53:42.291 INFO:teuthology.orchestra.run.smithi102.stdout:(8/34): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-09-11T07:53:42.450 INFO:teuthology.orchestra.run.smithi102.stdout:(9/34): ceph-osd-17.2.7-1623.gd0e6828a.el9.x86_ 27 MB/s | 15 MB 00:00 2024-09-11T07:53:42.475 INFO:teuthology.orchestra.run.smithi102.stdout:(10/34): python3-pycparser-2.20-6.el9.noarch.rp 599 kB/s | 135 kB 00:00 2024-09-11T07:53:42.508 INFO:teuthology.orchestra.run.smithi102.stdout:(11/34): python3-requests-2.25.1-8.el9.noarch.r 575 kB/s | 125 kB 00:00 2024-09-11T07:53:42.575 INFO:teuthology.orchestra.run.smithi102.stdout:(12/34): python3-urllib3-1.26.5-6.el9.noarch.rp 1.7 MB/s | 215 kB 00:00 2024-09-11T07:53:42.861 INFO:teuthology.orchestra.run.smithi146.stdout:(1/34): ceph-17.2.7-1623.gd0e6828a.el9.x86_64.r 53 kB/s | 6.5 kB 00:00 2024-09-11T07:53:43.028 INFO:teuthology.orchestra.run.smithi146.stdout:(2/34): ceph-mgr-17.2.7-1623.gd0e6828a.el9.x86_ 4.8 MB/s | 1.4 MB 00:00 2024-09-11T07:53:43.076 INFO:teuthology.orchestra.run.smithi102.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 61 kB/s | 35 kB 00:00 2024-09-11T07:53:43.095 INFO:teuthology.orchestra.run.smithi146.stdout:(3/34): ceph-mds-17.2.7-1623.gd0e6828a.el9.x86_ 5.8 MB/s | 2.1 MB 00:00 2024-09-11T07:53:43.170 INFO:teuthology.orchestra.run.smithi146.stdout:(4/34): ceph-mon-17.2.7-1623.gd0e6828a.el9.x86_ 13 MB/s | 3.9 MB 00:00 2024-09-11T07:53:43.195 INFO:teuthology.orchestra.run.smithi146.stdout:(5/34): ceph-mgr-modules-core-17.2.7-1623.gd0e6 2.3 MB/s | 240 kB 00:00 2024-09-11T07:53:43.201 INFO:teuthology.orchestra.run.smithi102.stdout:(14/34): python3-packaging-20.9-5.el9.noarch.rp 123 kB/s | 77 kB 00:00 2024-09-11T07:53:43.226 INFO:teuthology.orchestra.run.smithi102.stdout:(15/34): python3-mako-1.1.4-6.el9.noarch.rpm 229 kB/s | 172 kB 00:00 2024-09-11T07:53:43.268 INFO:teuthology.orchestra.run.smithi102.stdout:(16/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.0 MB/s | 43 kB 00:00 2024-09-11T07:53:43.293 INFO:teuthology.orchestra.run.smithi102.stdout:(17/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 6.7 MB/s | 172 kB 00:00 2024-09-11T07:53:43.327 INFO:teuthology.orchestra.run.smithi102.stdout:(18/34): python3-cherrypy-18.6.1-2.el9.noarch.r 11 MB/s | 358 kB 00:00 2024-09-11T07:53:43.329 INFO:teuthology.orchestra.run.smithi146.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-09-11T07:53:43.361 INFO:teuthology.orchestra.run.smithi102.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 411 kB/s | 11 kB 00:00 2024-09-11T07:53:43.371 INFO:teuthology.orchestra.run.smithi146.stdout:(7/34): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-11T07:53:43.386 INFO:teuthology.orchestra.run.smithi102.stdout:(20/34): python3-jaraco-classes-3.2.1-5.el9.noa 707 kB/s | 18 kB 00:00 2024-09-11T07:53:43.412 INFO:teuthology.orchestra.run.smithi102.stdout:(21/34): python3-jaraco-collections-3.0.0-8.el9 930 kB/s | 23 kB 00:00 2024-09-11T07:53:43.421 INFO:teuthology.orchestra.run.smithi146.stdout:(8/34): python3-cryptography-36.0.1-4.el9.x86_6 5.4 MB/s | 1.2 MB 00:00 2024-09-11T07:53:43.437 INFO:teuthology.orchestra.run.smithi102.stdout:(22/34): python3-jaraco-functools-3.5.0-2.el9.n 800 kB/s | 19 kB 00:00 2024-09-11T07:53:43.437 INFO:teuthology.orchestra.run.smithi146.stdout:(9/34): python3-pycparser-2.20-6.el9.noarch.rpm 2.0 MB/s | 135 kB 00:00 2024-09-11T07:53:43.462 INFO:teuthology.orchestra.run.smithi102.stdout:(23/34): python3-jaraco-text-3.2.0-6.el9.noarch 793 kB/s | 20 kB 00:00 2024-09-11T07:53:43.463 INFO:teuthology.orchestra.run.smithi146.stdout:(10/34): python3-requests-2.25.1-8.el9.noarch.r 2.9 MB/s | 125 kB 00:00 2024-09-11T07:53:43.488 INFO:teuthology.orchestra.run.smithi146.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-09-11T07:53:43.504 INFO:teuthology.orchestra.run.smithi102.stdout:(24/34): python3-logutils-0.3.5-21.el9.noarch.r 1.3 MB/s | 46 kB 00:00 2024-09-11T07:53:43.529 INFO:teuthology.orchestra.run.smithi102.stdout:(25/34): python3-more-itertools-8.12.0-2.el9.no 3.1 MB/s | 79 kB 00:00 2024-09-11T07:53:43.554 INFO:teuthology.orchestra.run.smithi102.stdout:(26/34): python3-pecan-1.4.2-3.el9.noarch.rpm 11 MB/s | 272 kB 00:00 2024-09-11T07:53:43.573 INFO:teuthology.orchestra.run.smithi102.stdout:(27/34): python3-portend-3.1.0-2.el9.noarch.rpm 936 kB/s | 16 kB 00:00 2024-09-11T07:53:43.598 INFO:teuthology.orchestra.run.smithi102.stdout:(28/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 3.5 MB/s | 90 kB 00:00 2024-09-11T07:53:43.615 INFO:teuthology.orchestra.run.smithi102.stdout:(29/34): python3-tempora-5.0.0-2.el9.noarch.rpm 2.1 MB/s | 36 kB 00:00 2024-09-11T07:53:43.641 INFO:teuthology.orchestra.run.smithi102.stdout:(30/34): python3-webob-1.8.8-2.el9.noarch.rpm 9.0 MB/s | 230 kB 00:00 2024-09-11T07:53:43.647 INFO:teuthology.orchestra.run.smithi146.stdout:(12/34): ceph-osd-17.2.7-1623.gd0e6828a.el9.x86 25 MB/s | 15 MB 00:00 2024-09-11T07:53:43.674 INFO:teuthology.orchestra.run.smithi102.stdout:(31/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 13 MB/s | 427 kB 00:00 2024-09-11T07:53:43.699 INFO:teuthology.orchestra.run.smithi102.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 804 kB/s | 20 kB 00:00 2024-09-11T07:53:43.722 INFO:teuthology.orchestra.run.smithi146.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 149 kB/s | 35 kB 00:00 2024-09-11T07:53:43.741 INFO:teuthology.orchestra.run.smithi102.stdout:(33/34): python3-pytz-2021.1-5.el9.noarch.rpm 77 kB/s | 51 kB 00:00 2024-09-11T07:53:43.747 INFO:teuthology.orchestra.run.smithi146.stdout:(14/34): python3-mako-1.1.4-6.el9.noarch.rpm 606 kB/s | 172 kB 00:00 2024-09-11T07:53:43.797 INFO:teuthology.orchestra.run.smithi146.stdout:(15/34): python3-toml-0.10.2-6.el9.noarch.rpm 833 kB/s | 42 kB 00:00 2024-09-11T07:53:43.823 INFO:teuthology.orchestra.run.smithi146.stdout:(16/34): python3-packaging-20.9-5.el9.noarch.rp 440 kB/s | 77 kB 00:00 2024-09-11T07:53:43.839 INFO:teuthology.orchestra.run.smithi146.stdout:(17/34): python3-pytz-2021.1-5.el9.noarch.rpm 435 kB/s | 51 kB 00:00 2024-09-11T07:53:43.858 INFO:teuthology.orchestra.run.smithi102.stdout:(34/34): python3-toml-0.10.2-6.el9.noarch.rpm 63 kB/s | 42 kB 00:00 2024-09-11T07:53:43.863 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:53:43.864 INFO:teuthology.orchestra.run.smithi102.stdout:Total 9.9 MB/s | 27 MB 00:02 2024-09-11T07:53:43.882 INFO:teuthology.orchestra.run.smithi146.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 518 kB/s | 43 kB 00:00 2024-09-11T07:53:43.898 INFO:teuthology.orchestra.run.smithi146.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 723 kB/s | 11 kB 00:00 2024-09-11T07:53:43.915 INFO:teuthology.orchestra.run.smithi146.stdout:(20/34): python3-jaraco-classes-3.2.1-5.el9.noa 1.0 MB/s | 18 kB 00:00 2024-09-11T07:53:43.941 INFO:teuthology.orchestra.run.smithi146.stdout:(21/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-09-11T07:53:43.958 INFO:teuthology.orchestra.run.smithi146.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 552 kB/s | 23 kB 00:00 2024-09-11T07:53:43.974 INFO:teuthology.orchestra.run.smithi146.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 583 kB/s | 19 kB 00:00 2024-09-11T07:53:43.992 INFO:teuthology.orchestra.run.smithi146.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 586 kB/s | 20 kB 00:00 2024-09-11T07:53:43.995 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-11T07:53:44.008 INFO:teuthology.orchestra.run.smithi146.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 1.3 MB/s | 46 kB 00:00 2024-09-11T07:53:44.033 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-11T07:53:44.033 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-11T07:53:44.033 INFO:teuthology.orchestra.run.smithi146.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2024-09-11T07:53:44.059 INFO:teuthology.orchestra.run.smithi146.stdout:(27/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.6 MB/s | 358 kB 00:00 2024-09-11T07:53:44.076 INFO:teuthology.orchestra.run.smithi146.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 390 kB/s | 16 kB 00:00 2024-09-11T07:53:44.101 INFO:teuthology.orchestra.run.smithi146.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.1 MB/s | 90 kB 00:00 2024-09-11T07:53:44.126 INFO:teuthology.orchestra.run.smithi146.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 713 kB/s | 36 kB 00:00 2024-09-11T07:53:44.151 INFO:teuthology.orchestra.run.smithi146.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-11T07:53:44.186 INFO:teuthology.orchestra.run.smithi146.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 594 kB/s | 20 kB 00:00 2024-09-11T07:53:44.210 INFO:teuthology.orchestra.run.smithi146.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-11T07:53:44.236 INFO:teuthology.orchestra.run.smithi146.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-11T07:53:44.240 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:53:44.240 INFO:teuthology.orchestra.run.smithi146.stdout:Total 13 MB/s | 27 MB 00:02 2024-09-11T07:53:44.366 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-11T07:53:44.405 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-11T07:53:44.405 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-11T07:53:44.420 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-11T07:53:44.420 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-11T07:53:44.779 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-11T07:53:44.780 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-11T07:53:45.653 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-11T07:53:45.720 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-11T07:53:45.752 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-11T07:53:45.784 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-11T07:53:45.885 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-11T07:53:45.963 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-11T07:53:46.004 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-11T07:53:46.039 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-11T07:53:46.047 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-11T07:53:46.069 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-11T07:53:46.103 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-11T07:53:46.105 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-11T07:53:46.137 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-11T07:53:46.137 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-11T07:53:46.170 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-11T07:53:46.177 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-11T07:53:46.212 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-11T07:53:46.255 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-11T07:53:46.262 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-11T07:53:46.304 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-11T07:53:46.335 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T07:53:46.347 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-11T07:53:46.389 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-11T07:53:46.409 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-11T07:53:46.424 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-11T07:53:46.454 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-11T07:53:46.488 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-11T07:53:46.522 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-11T07:53:46.549 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-11T07:53:46.562 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-11T07:53:46.606 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-11T07:53:46.634 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-11T07:53:46.640 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-11T07:53:46.687 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-11T07:53:46.688 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-11T07:53:46.720 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T07:53:46.735 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-11T07:53:46.792 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-11T07:53:46.793 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-11T07:53:46.847 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-11T07:53:46.957 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-11T07:53:47.033 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-11T07:53:47.045 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-11T07:53:47.089 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-11T07:53:47.098 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-11T07:53:47.137 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-11T07:53:47.158 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-11T07:53:47.194 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-11T07:53:47.246 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-11T07:53:47.249 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-11T07:53:47.288 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-11T07:53:47.413 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-11T07:53:47.435 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-11T07:53:47.482 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-11T07:53:47.562 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-11T07:53:47.566 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 29/34 2024-09-11T07:53:47.599 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T07:53:47.624 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T07:53:47.625 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:53:47.625 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T07:53:47.625 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T07:53:47.625 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T07:53:47.625 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:47.650 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-11T07:53:47.699 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-11T07:53:47.816 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-11T07:53:47.969 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 29/34 2024-09-11T07:53:48.008 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T07:53:48.034 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 30/34 2024-09-11T07:53:48.034 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:53:48.034 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T07:53:48.034 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T07:53:48.034 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T07:53:48.034 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:49.000 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T07:53:49.029 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T07:53:49.029 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:53:49.030 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T07:53:49.030 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T07:53:49.030 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T07:53:49.030 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:49.401 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T07:53:49.417 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T07:53:49.431 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 31/34 2024-09-11T07:53:49.431 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:53:49.431 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T07:53:49.431 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T07:53:49.431 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T07:53:49.431 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:49.442 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T07:53:49.442 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:53:49.442 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T07:53:49.442 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T07:53:49.442 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T07:53:49.442 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:49.644 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T07:53:49.668 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T07:53:49.668 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:53:49.668 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T07:53:49.668 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T07:53:49.668 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T07:53:49.668 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:49.732 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T07:53:49.810 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T07:53:49.834 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 32/34 2024-09-11T07:53:49.835 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:53:49.835 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T07:53:49.835 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T07:53:49.835 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T07:53:49.835 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:50.036 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T07:53:50.060 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 33/34 2024-09-11T07:53:50.060 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:53:50.060 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T07:53:50.060 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T07:53:50.060 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T07:53:50.060 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:50.124 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T07:53:51.446 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T07:53:51.446 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/34 2024-09-11T07:53:51.447 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/34 2024-09-11T07:53:51.447 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/34 2024-09-11T07:53:51.447 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/34 2024-09-11T07:53:51.447 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/34 2024-09-11T07:53:51.447 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 6/34 2024-09-11T07:53:51.447 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-11T07:53:51.447 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-11T07:53:51.447 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-11T07:53:51.447 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-11T07:53:51.447 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-11T07:53:51.447 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-11T07:53:51.447 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-11T07:53:51.447 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-11T07:53:51.448 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T07:53:51.448 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-11T07:53:51.448 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-11T07:53:51.448 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-11T07:53:51.448 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-11T07:53:51.448 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-11T07:53:51.448 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-11T07:53:51.448 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-11T07:53:51.448 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-11T07:53:51.448 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-11T07:53:51.448 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-11T07:53:51.449 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-11T07:53:51.449 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-11T07:53:51.450 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-11T07:53:51.450 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-11T07:53:51.450 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-11T07:53:51.450 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-11T07:53:51.450 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-11T07:53:51.450 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-11T07:53:51.855 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 34/34 2024-09-11T07:53:51.855 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/34 2024-09-11T07:53:51.855 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/34 2024-09-11T07:53:51.855 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/34 2024-09-11T07:53:51.856 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/34 2024-09-11T07:53:51.856 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/34 2024-09-11T07:53:51.856 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9. 6/34 2024-09-11T07:53:51.856 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-11T07:53:51.856 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-11T07:53:51.856 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-11T07:53:51.856 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-11T07:53:51.856 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-11T07:53:51.856 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-11T07:53:51.856 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-11T07:53:51.856 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-11T07:53:51.856 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-11T07:53:51.857 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-11T07:53:51.857 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-11T07:53:51.857 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-11T07:53:51.857 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-11T07:53:51.857 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-11T07:53:51.857 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-11T07:53:51.857 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-11T07:53:51.857 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-11T07:53:51.857 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-11T07:53:51.857 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-11T07:53:51.857 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-11T07:53:51.857 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-11T07:53:51.858 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-11T07:53:51.858 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-11T07:53:51.858 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-11T07:53:51.858 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-11T07:53:51.859 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-11T07:53:51.859 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-11T07:53:52.414 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-11T07:53:52.414 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:52.414 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-11T07:53:52.414 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:52.415 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:52.415 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:52.415 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:53:52.415 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:52.415 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:52.415 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T07:53:52.415 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T07:53:52.415 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T07:53:52.415 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T07:53:52.415 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T07:53:52.415 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T07:53:52.415 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T07:53:52.415 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T07:53:52.415 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T07:53:52.416 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T07:53:52.416 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T07:53:52.416 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T07:53:52.416 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T07:53:52.416 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T07:53:52.416 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T07:53:52.416 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T07:53:52.416 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T07:53:52.416 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T07:53:52.416 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T07:53:52.416 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T07:53:52.416 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T07:53:52.417 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T07:53:52.417 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T07:53:52.417 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T07:53:52.417 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T07:53:52.417 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T07:53:52.417 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T07:53:52.417 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T07:53:52.417 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:52.417 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:53:52.786 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-base 2024-09-11T07:53:52.812 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-11T07:53:52.812 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:52.812 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-11T07:53:52.813 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:52.813 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:52.813 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:52.813 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:53:52.813 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:52.813 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:52.813 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T07:53:52.813 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T07:53:52.813 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T07:53:52.813 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T07:53:52.813 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T07:53:52.813 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T07:53:52.814 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T07:53:52.814 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T07:53:52.814 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T07:53:52.814 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T07:53:52.814 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T07:53:52.814 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T07:53:52.814 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T07:53:52.814 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T07:53:52.814 INFO:teuthology.orchestra.run.smithi146.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T07:53:52.814 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T07:53:52.814 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T07:53:52.814 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T07:53:52.814 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T07:53:52.814 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T07:53:52.815 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T07:53:52.815 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T07:53:52.815 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T07:53:52.815 INFO:teuthology.orchestra.run.smithi146.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T07:53:52.815 INFO:teuthology.orchestra.run.smithi146.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T07:53:52.815 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T07:53:52.815 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T07:53:52.815 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T07:53:52.815 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:52.815 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:53:53.211 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-base 2024-09-11T07:53:53.361 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:51 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:53:53.452 INFO:teuthology.orchestra.run.smithi102.stdout:Package ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:53:53.493 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:53:53.494 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-11T07:53:53.494 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:53:53.565 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install cephadm 2024-09-11T07:53:53.779 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:00:50 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:53:53.868 INFO:teuthology.orchestra.run.smithi146.stdout:Package ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:53:53.911 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:53:53.912 INFO:teuthology.orchestra.run.smithi146.stdout:Nothing to do. 2024-09-11T07:53:53.912 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:53:53.984 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install cephadm 2024-09-11T07:53:54.119 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:52 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:53:54.252 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:53:54.253 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:53:54.253 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-11T07:53:54.253 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:53:54.253 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-11T07:53:54.253 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 83 k 2024-09-11T07:53:54.253 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:54.253 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-11T07:53:54.253 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:53:54.253 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-11T07:53:54.253 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:54.254 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 83 k 2024-09-11T07:53:54.254 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 375 k 2024-09-11T07:53:54.254 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-11T07:53:54.428 INFO:teuthology.orchestra.run.smithi102.stdout:cephadm-17.2.7-1623.gd0e6828a.el9.noarch.rpm 476 kB/s | 83 kB 00:00 2024-09-11T07:53:54.428 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:53:54.428 INFO:teuthology.orchestra.run.smithi102.stdout:Total 471 kB/s | 83 kB 00:00 2024-09-11T07:53:54.428 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-11T07:53:54.433 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-11T07:53:54.433 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-11T07:53:54.435 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-11T07:53:54.436 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-11T07:53:54.509 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-11T07:53:54.549 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:00:51 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:53:54.682 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:53:54.683 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:53:54.683 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-09-11T07:53:54.683 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:53:54.683 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-11T07:53:54.683 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 83 k 2024-09-11T07:53:54.683 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:54.683 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-11T07:53:54.683 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:53:54.683 INFO:teuthology.orchestra.run.smithi146.stdout:Install 1 Package 2024-09-11T07:53:54.684 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:54.684 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 83 k 2024-09-11T07:53:54.684 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 375 k 2024-09-11T07:53:54.684 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-11T07:53:54.867 INFO:teuthology.orchestra.run.smithi146.stdout:cephadm-17.2.7-1623.gd0e6828a.el9.noarch.rpm 452 kB/s | 83 kB 00:00 2024-09-11T07:53:54.868 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:53:54.868 INFO:teuthology.orchestra.run.smithi146.stdout:Total 448 kB/s | 83 kB 00:00 2024-09-11T07:53:54.868 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-11T07:53:54.873 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-11T07:53:54.873 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-11T07:53:54.875 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-11T07:53:54.875 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-11T07:53:54.894 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:53:54.952 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-11T07:53:54.997 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:53:55.330 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:53:55.391 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:53:55.470 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:53:55.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:53:55.595 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:55.595 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-11T07:53:55.595 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:53:55.595 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:55.595 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:53:55.800 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-immutable-object-cache 2024-09-11T07:53:55.831 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:53:56.080 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:53:56.080 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:56.080 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-11T07:53:56.080 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:53:56.080 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:56.080 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:53:56.305 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-immutable-object-cache 2024-09-11T07:53:56.372 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:54 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:53:56.503 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:53:56.504 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:53:56.504 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-11T07:53:56.504 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:53:56.504 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-11T07:53:56.504 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 137 k 2024-09-11T07:53:56.504 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:56.505 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-11T07:53:56.505 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:53:56.505 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-11T07:53:56.505 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:56.505 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 137 k 2024-09-11T07:53:56.505 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 379 k 2024-09-11T07:53:56.505 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-11T07:53:56.692 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-immutable-object-cache-17.2.7-1623.gd0e682 736 kB/s | 137 kB 00:00 2024-09-11T07:53:56.693 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:53:56.693 INFO:teuthology.orchestra.run.smithi102.stdout:Total 727 kB/s | 137 kB 00:00 2024-09-11T07:53:56.694 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-11T07:53:56.701 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-11T07:53:56.701 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-11T07:53:56.758 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-11T07:53:56.759 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-11T07:53:56.884 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:00:53 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:53:56.925 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-11T07:53:57.023 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:53:57.024 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:53:57.024 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repo Size 2024-09-11T07:53:57.024 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:53:57.024 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-11T07:53:57.024 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 137 k 2024-09-11T07:53:57.024 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:57.024 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-11T07:53:57.024 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:53:57.025 INFO:teuthology.orchestra.run.smithi146.stdout:Install 1 Package 2024-09-11T07:53:57.025 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:57.025 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 137 k 2024-09-11T07:53:57.025 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 379 k 2024-09-11T07:53:57.025 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-11T07:53:57.026 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:53:57.051 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:53:57.052 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:53:57.052 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T07:53:57.052 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:57.215 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-immutable-object-cache-17.2.7-1623.gd0e682 725 kB/s | 137 kB 00:00 2024-09-11T07:53:57.215 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:53:57.215 INFO:teuthology.orchestra.run.smithi146.stdout:Total 721 kB/s | 137 kB 00:00 2024-09-11T07:53:57.215 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-11T07:53:57.222 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-11T07:53:57.222 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-11T07:53:57.280 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-11T07:53:57.280 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-11T07:53:57.435 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-11T07:53:57.498 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:53:57.524 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:53:57.524 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:53:57.524 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T07:53:57.524 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:57.978 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:53:57.978 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:57.978 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-11T07:53:57.978 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:57.978 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:57.978 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:53:58.217 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mgr 2024-09-11T07:53:58.456 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a. 1/1 2024-09-11T07:53:58.457 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:58.457 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-11T07:53:58.457 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:53:58.457 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:53:58.457 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:53:58.695 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-mgr 2024-09-11T07:53:58.786 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:56 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:53:58.879 INFO:teuthology.orchestra.run.smithi102.stdout:Package ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:53:58.922 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:53:58.923 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-11T07:53:58.924 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:53:58.993 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mgr-dashboard 2024-09-11T07:53:59.274 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:00:56 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:53:59.368 INFO:teuthology.orchestra.run.smithi146.stdout:Package ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:53:59.410 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:53:59.411 INFO:teuthology.orchestra.run.smithi146.stdout:Nothing to do. 2024-09-11T07:53:59.411 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:53:59.483 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-mgr-dashboard 2024-09-11T07:53:59.562 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:57 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:53:59.696 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:53:59.697 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:53:59.697 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-11T07:53:59.697 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:53:59.697 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-11T07:53:59.697 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 3.4 M 2024-09-11T07:53:59.697 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-11T07:53:59.697 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 24 k 2024-09-11T07:53:59.698 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 15 k 2024-09-11T07:53:59.698 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-11T07:53:59.698 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T07:53:59.698 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T07:53:59.698 INFO:teuthology.orchestra.run.smithi102.stdout:Installing weak dependencies: 2024-09-11T07:53:59.698 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-11T07:53:59.698 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:59.699 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-11T07:53:59.699 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:53:59.699 INFO:teuthology.orchestra.run.smithi102.stdout:Install 7 Packages 2024-09-11T07:53:59.699 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:53:59.699 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 3.7 M 2024-09-11T07:53:59.699 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 80 M 2024-09-11T07:53:59.700 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-11T07:54:00.068 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:00:57 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:54:00.088 INFO:teuthology.orchestra.run.smithi102.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1623.gd0e 171 kB/s | 24 kB 00:00 2024-09-11T07:54:00.105 INFO:teuthology.orchestra.run.smithi102.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1623.gd0e6 92 kB/s | 15 kB 00:00 2024-09-11T07:54:00.147 INFO:teuthology.orchestra.run.smithi102.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 154 kB/s | 9.0 kB 00:00 2024-09-11T07:54:00.172 INFO:teuthology.orchestra.run.smithi102.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-11T07:54:00.197 INFO:teuthology.orchestra.run.smithi102.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 442 kB/s | 41 kB 00:00 2024-09-11T07:54:00.201 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:54:00.202 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:00.202 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-09-11T07:54:00.202 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:00.202 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-11T07:54:00.203 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 3.4 M 2024-09-11T07:54:00.203 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-09-11T07:54:00.203 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 24 k 2024-09-11T07:54:00.203 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 15 k 2024-09-11T07:54:00.203 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-11T07:54:00.203 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T07:54:00.203 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T07:54:00.203 INFO:teuthology.orchestra.run.smithi146.stdout:Installing weak dependencies: 2024-09-11T07:54:00.203 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-11T07:54:00.203 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:00.203 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-11T07:54:00.203 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:00.203 INFO:teuthology.orchestra.run.smithi146.stdout:Install 7 Packages 2024-09-11T07:54:00.204 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:00.204 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 3.7 M 2024-09-11T07:54:00.204 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 80 M 2024-09-11T07:54:00.204 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-11T07:54:00.222 INFO:teuthology.orchestra.run.smithi102.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.6 MB/s | 188 kB 00:00 2024-09-11T07:54:00.314 INFO:teuthology.orchestra.run.smithi102.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1623.gd0e6828a 9.1 MB/s | 3.4 MB 00:00 2024-09-11T07:54:00.317 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:54:00.318 INFO:teuthology.orchestra.run.smithi102.stdout:Total 5.9 MB/s | 3.7 MB 00:00 2024-09-11T07:54:00.318 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-11T07:54:00.326 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-11T07:54:00.326 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-11T07:54:00.407 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-11T07:54:00.408 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-11T07:54:00.534 INFO:teuthology.orchestra.run.smithi146.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1623.gd0e 161 kB/s | 24 kB 00:00 2024-09-11T07:54:00.559 INFO:teuthology.orchestra.run.smithi146.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1623.gd0e6 83 kB/s | 15 kB 00:00 2024-09-11T07:54:00.601 INFO:teuthology.orchestra.run.smithi146.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 135 kB/s | 9.0 kB 00:00 2024-09-11T07:54:00.626 INFO:teuthology.orchestra.run.smithi146.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-11T07:54:00.651 INFO:teuthology.orchestra.run.smithi146.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 444 kB/s | 41 kB 00:00 2024-09-11T07:54:00.677 INFO:teuthology.orchestra.run.smithi146.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-11T07:54:00.707 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-11T07:54:00.774 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-11T07:54:00.802 INFO:teuthology.orchestra.run.smithi146.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1623.gd0e6828a 8.0 MB/s | 3.4 MB 00:00 2024-09-11T07:54:00.805 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:54:00.805 INFO:teuthology.orchestra.run.smithi146.stdout:Total 6.1 MB/s | 3.7 MB 00:00 2024-09-11T07:54:00.805 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-11T07:54:00.808 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-11T07:54:00.814 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-11T07:54:00.814 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-11T07:54:00.863 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-11T07:54:00.899 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-11T07:54:00.899 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-11T07:54:00.903 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-11T07:54:00.932 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 5/7 2024-09-11T07:54:01.205 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-11T07:54:01.272 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-11T07:54:01.306 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-11T07:54:01.360 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-11T07:54:01.401 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-11T07:54:01.430 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 5/7 2024-09-11T07:54:01.929 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 6/7 2024-09-11T07:54:01.961 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T07:54:02.414 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 6/7 2024-09-11T07:54:02.458 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T07:54:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T07:54:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 1/7 2024-09-11T07:54:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 2/7 2024-09-11T07:54:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 3/7 2024-09-11T07:54:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-11T07:54:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-11T07:54:03.124 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-11T07:54:03.411 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-11T07:54:03.411 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:03.411 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-11T07:54:03.411 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:54:03.411 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:54:03.411 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:54:03.412 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-11T07:54:03.412 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-11T07:54:03.412 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T07:54:03.412 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T07:54:03.412 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:03.412 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:54:03.505 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 7/7 2024-09-11T07:54:03.506 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9. 1/7 2024-09-11T07:54:03.506 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarc 2/7 2024-09-11T07:54:03.506 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.n 3/7 2024-09-11T07:54:03.506 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-11T07:54:03.506 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-11T07:54:03.506 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-11T07:54:03.634 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-11T07:54:03.791 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-11T07:54:03.791 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:03.791 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-11T07:54:03.791 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:54:03.791 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:54:03.791 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:54:03.791 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-11T07:54:03.792 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-11T07:54:03.792 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T07:54:03.792 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T07:54:03.792 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:03.792 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:54:04.018 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-11T07:54:04.189 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:02 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:54:04.321 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:54:04.322 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-09-11T07:54:04.322 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-11T07:54:04.322 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-09-11T07:54:04.323 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-11T07:54:04.323 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 7.4 M 2024-09-11T07:54:04.323 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-11T07:54:04.323 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T07:54:04.323 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T07:54:04.323 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T07:54:04.323 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T07:54:04.323 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T07:54:04.323 INFO:teuthology.orchestra.run.smithi102.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T07:54:04.323 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T07:54:04.323 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T07:54:04.323 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T07:54:04.323 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T07:54:04.324 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T07:54:04.324 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:04.324 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-11T07:54:04.324 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-09-11T07:54:04.324 INFO:teuthology.orchestra.run.smithi102.stdout:Install 12 Packages 2024-09-11T07:54:04.324 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:04.324 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 38 M 2024-09-11T07:54:04.325 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 212 M 2024-09-11T07:54:04.325 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-11T07:54:04.580 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:01:01 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:54:04.713 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:54:04.714 INFO:teuthology.orchestra.run.smithi146.stdout:======================================================================================= 2024-09-11T07:54:04.715 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-09-11T07:54:04.715 INFO:teuthology.orchestra.run.smithi146.stdout:======================================================================================= 2024-09-11T07:54:04.715 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-11T07:54:04.715 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 7.4 M 2024-09-11T07:54:04.715 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-09-11T07:54:04.715 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T07:54:04.715 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T07:54:04.715 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T07:54:04.715 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T07:54:04.715 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T07:54:04.715 INFO:teuthology.orchestra.run.smithi146.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T07:54:04.715 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T07:54:04.716 INFO:teuthology.orchestra.run.smithi146.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T07:54:04.716 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T07:54:04.716 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T07:54:04.716 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T07:54:04.716 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:04.716 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-11T07:54:04.716 INFO:teuthology.orchestra.run.smithi146.stdout:======================================================================================= 2024-09-11T07:54:04.716 INFO:teuthology.orchestra.run.smithi146.stdout:Install 12 Packages 2024-09-11T07:54:04.716 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:04.717 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 38 M 2024-09-11T07:54:04.717 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 212 M 2024-09-11T07:54:04.717 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-11T07:54:05.028 INFO:teuthology.orchestra.run.smithi146.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 191 kB 00:00 2024-09-11T07:54:05.078 INFO:teuthology.orchestra.run.smithi146.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.2 MB/s | 801 kB 00:00 2024-09-11T07:54:05.170 INFO:teuthology.orchestra.run.smithi146.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 231 kB/s | 33 kB 00:00 2024-09-11T07:54:05.221 INFO:teuthology.orchestra.run.smithi146.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 363 kB/s | 18 kB 00:00 2024-09-11T07:54:05.271 INFO:teuthology.orchestra.run.smithi146.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 768 kB/s | 38 kB 00:00 2024-09-11T07:54:05.415 INFO:teuthology.orchestra.run.smithi102.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 343 kB/s | 191 kB 00:00 2024-09-11T07:54:05.699 INFO:teuthology.orchestra.run.smithi102.stdout:(2/12): ceph-mgr-diskprediction-local-17.2.7-16 8.8 MB/s | 7.4 MB 00:00 2024-09-11T07:54:05.732 INFO:teuthology.orchestra.run.smithi102.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 916 kB/s | 801 kB 00:00 2024-09-11T07:54:05.913 INFO:teuthology.orchestra.run.smithi146.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.6 MB/s | 3.0 MB 00:00 2024-09-11T07:54:06.047 INFO:teuthology.orchestra.run.smithi146.stdout:(7/12): ceph-mgr-diskprediction-local-17.2.7-16 6.4 MB/s | 7.4 MB 00:01 2024-09-11T07:54:06.049 INFO:teuthology.orchestra.run.smithi102.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 52 kB/s | 33 kB 00:00 2024-09-11T07:54:06.072 INFO:teuthology.orchestra.run.smithi146.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.5 MB/s | 245 kB 00:00 2024-09-11T07:54:06.164 INFO:teuthology.orchestra.run.smithi146.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.7 MB/s | 250 kB 00:00 2024-09-11T07:54:06.241 INFO:teuthology.orchestra.run.smithi102.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 36 kB/s | 18 kB 00:00 2024-09-11T07:54:06.247 INFO:teuthology.orchestra.run.smithi146.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 5.1 MB/s | 4.9 MB 00:00 2024-09-11T07:54:06.650 INFO:teuthology.orchestra.run.smithi102.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 64 kB/s | 38 kB 00:00 2024-09-11T07:54:06.875 INFO:teuthology.orchestra.run.smithi102.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.6 MB/s | 3.0 MB 00:01 2024-09-11T07:54:07.359 INFO:teuthology.orchestra.run.smithi102.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 4.4 MB/s | 4.9 MB 00:01 2024-09-11T07:54:07.364 INFO:teuthology.orchestra.run.smithi146.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.9 MB/s | 5.1 MB 00:01 2024-09-11T07:54:07.492 INFO:teuthology.orchestra.run.smithi102.stdout:(9/12): python3-devel-3.9.19-8.el9.x86_64.rpm 291 kB/s | 245 kB 00:00 2024-09-11T07:54:08.134 INFO:teuthology.orchestra.run.smithi102.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 323 kB/s | 250 kB 00:00 2024-09-11T07:54:08.217 INFO:teuthology.orchestra.run.smithi102.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.8 MB/s | 5.1 MB 00:01 2024-09-11T07:54:09.301 INFO:teuthology.orchestra.run.smithi102.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 9.0 MB/s | 16 MB 00:01 2024-09-11T07:54:09.302 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:54:09.302 INFO:teuthology.orchestra.run.smithi102.stdout:Total 7.7 MB/s | 38 MB 00:04 2024-09-11T07:54:09.659 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-11T07:54:09.686 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-11T07:54:09.687 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-11T07:54:10.203 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-11T07:54:10.204 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-11T07:54:10.473 INFO:teuthology.orchestra.run.smithi146.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.8 MB/s | 16 MB 00:04 2024-09-11T07:54:10.473 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:54:10.474 INFO:teuthology.orchestra.run.smithi146.stdout:Total 6.6 MB/s | 38 MB 00:05 2024-09-11T07:54:10.829 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-11T07:54:10.855 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-11T07:54:10.855 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-11T07:54:10.992 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-11T07:54:11.063 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-11T07:54:11.107 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-11T07:54:11.218 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-11T07:54:11.258 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-11T07:54:11.387 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-11T07:54:11.387 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-11T07:54:11.450 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-11T07:54:11.483 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-11T07:54:11.574 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-11T07:54:12.145 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-11T07:54:12.217 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-11T07:54:12.241 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-11T07:54:12.262 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-11T07:54:12.364 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-11T07:54:12.365 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-11T07:54:12.405 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-11T07:54:12.591 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-11T07:54:12.620 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-11T07:54:12.711 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-11T07:54:13.356 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-11T07:54:13.476 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-11T07:54:13.655 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T07:54:14.715 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-11T07:54:14.730 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T07:54:14.741 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T07:54:15.770 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-11T07:54:15.810 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T07:54:17.240 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T07:54:17.240 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 1/12 2024-09-11T07:54:17.240 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-11T07:54:17.240 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-11T07:54:17.240 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-11T07:54:17.240 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-11T07:54:17.240 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-11T07:54:17.240 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-11T07:54:17.240 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-11T07:54:17.241 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-11T07:54:17.241 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T07:54:17.241 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-11T07:54:17.660 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-11T07:54:17.661 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:17.661 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-11T07:54:17.661 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:54:17.661 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T07:54:17.661 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T07:54:17.661 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T07:54:17.661 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T07:54:17.661 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T07:54:17.661 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T07:54:17.661 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T07:54:17.662 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T07:54:17.662 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T07:54:17.662 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T07:54:17.662 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T07:54:17.662 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:17.662 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:54:18.055 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mgr-rook 2024-09-11T07:54:18.468 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 12/12 2024-09-11T07:54:18.468 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e68 1/12 2024-09-11T07:54:18.468 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-11T07:54:18.468 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-11T07:54:18.468 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-11T07:54:18.469 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-11T07:54:18.469 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-11T07:54:18.469 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-11T07:54:18.469 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-11T07:54:18.469 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-11T07:54:18.469 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T07:54:18.469 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-11T07:54:18.641 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:16 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:54:18.783 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:54:18.784 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:18.784 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-11T07:54:18.785 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:18.785 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-11T07:54:18.785 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 47 k 2024-09-11T07:54:18.785 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-11T07:54:18.785 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T07:54:18.785 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T07:54:18.785 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T07:54:18.785 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T07:54:18.785 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T07:54:18.785 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T07:54:18.785 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T07:54:18.785 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T07:54:18.786 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T07:54:18.786 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T07:54:18.786 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T07:54:18.786 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T07:54:18.786 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:18.786 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-11T07:54:18.786 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:18.786 INFO:teuthology.orchestra.run.smithi102.stdout:Install 13 Packages 2024-09-11T07:54:18.786 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:18.787 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 2.2 M 2024-09-11T07:54:18.787 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 26 M 2024-09-11T07:54:18.787 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-11T07:54:18.872 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-11T07:54:18.872 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:18.872 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-11T07:54:18.872 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:54:18.873 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T07:54:18.873 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T07:54:18.873 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T07:54:18.873 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T07:54:18.873 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T07:54:18.873 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T07:54:18.873 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T07:54:18.873 INFO:teuthology.orchestra.run.smithi146.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T07:54:18.873 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T07:54:18.873 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T07:54:18.873 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T07:54:18.873 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:18.873 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:54:19.183 INFO:teuthology.orchestra.run.smithi102.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 159 kB/s | 19 kB 00:00 2024-09-11T07:54:19.208 INFO:teuthology.orchestra.run.smithi102.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 176 kB/s | 26 kB 00:00 2024-09-11T07:54:19.233 INFO:teuthology.orchestra.run.smithi102.stdout:(3/13): ceph-mgr-rook-17.2.7-1623.gd0e6828a.el9 278 kB/s | 47 kB 00:00 2024-09-11T07:54:19.254 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-mgr-rook 2024-09-11T07:54:19.300 INFO:teuthology.orchestra.run.smithi102.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.9 MB/s | 222 kB 00:00 2024-09-11T07:54:19.326 INFO:teuthology.orchestra.run.smithi102.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-11T07:54:19.359 INFO:teuthology.orchestra.run.smithi102.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 913 kB/s | 54 kB 00:00 2024-09-11T07:54:19.393 INFO:teuthology.orchestra.run.smithi102.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 480 kB/s | 32 kB 00:00 2024-09-11T07:54:19.418 INFO:teuthology.orchestra.run.smithi102.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 248 kB/s | 14 kB 00:00 2024-09-11T07:54:19.443 INFO:teuthology.orchestra.run.smithi102.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 4.4 MB/s | 220 kB 00:00 2024-09-11T07:54:19.477 INFO:teuthology.orchestra.run.smithi102.stdout:(10/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.1 MB/s | 279 kB 00:00 2024-09-11T07:54:19.502 INFO:teuthology.orchestra.run.smithi102.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-11T07:54:19.553 INFO:teuthology.orchestra.run.smithi102.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 7.6 MB/s | 1.0 MB 00:00 2024-09-11T07:54:19.578 INFO:teuthology.orchestra.run.smithi102.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 893 kB/s | 90 kB 00:00 2024-09-11T07:54:19.580 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:54:19.580 INFO:teuthology.orchestra.run.smithi102.stdout:Total 2.8 MB/s | 2.2 MB 00:00 2024-09-11T07:54:19.645 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-11T07:54:19.660 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-11T07:54:19.661 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-11T07:54:19.752 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-11T07:54:19.752 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-11T07:54:19.839 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:01:16 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:54:19.981 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:54:19.983 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:19.983 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-09-11T07:54:19.983 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:19.983 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-11T07:54:19.983 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 47 k 2024-09-11T07:54:19.983 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-09-11T07:54:19.983 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T07:54:19.983 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T07:54:19.983 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T07:54:19.983 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T07:54:19.983 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T07:54:19.984 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T07:54:19.984 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T07:54:19.984 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T07:54:19.984 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T07:54:19.984 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T07:54:19.984 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T07:54:19.984 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T07:54:19.984 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:19.984 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-11T07:54:19.984 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:19.984 INFO:teuthology.orchestra.run.smithi146.stdout:Install 13 Packages 2024-09-11T07:54:19.985 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:19.985 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 2.2 M 2024-09-11T07:54:19.985 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 26 M 2024-09-11T07:54:19.985 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-11T07:54:20.268 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-11T07:54:20.277 INFO:teuthology.orchestra.run.smithi146.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 288 kB/s | 26 kB 00:00 2024-09-11T07:54:20.302 INFO:teuthology.orchestra.run.smithi146.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 168 kB/s | 19 kB 00:00 2024-09-11T07:54:20.361 INFO:teuthology.orchestra.run.smithi146.stdout:(3/13): ceph-mgr-rook-17.2.7-1623.gd0e6828a.el9 275 kB/s | 47 kB 00:00 2024-09-11T07:54:20.402 INFO:teuthology.orchestra.run.smithi146.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.7 MB/s | 222 kB 00:00 2024-09-11T07:54:20.425 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-11T07:54:20.428 INFO:teuthology.orchestra.run.smithi146.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-11T07:54:20.445 INFO:teuthology.orchestra.run.smithi146.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.2 MB/s | 54 kB 00:00 2024-09-11T07:54:20.479 INFO:teuthology.orchestra.run.smithi146.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 641 kB/s | 32 kB 00:00 2024-09-11T07:54:20.487 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-11T07:54:20.504 INFO:teuthology.orchestra.run.smithi146.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 241 kB/s | 14 kB 00:00 2024-09-11T07:54:20.537 INFO:teuthology.orchestra.run.smithi146.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.5 MB/s | 279 kB 00:00 2024-09-11T07:54:20.551 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-11T07:54:20.563 INFO:teuthology.orchestra.run.smithi146.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.6 MB/s | 220 kB 00:00 2024-09-11T07:54:20.589 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-11T07:54:20.621 INFO:teuthology.orchestra.run.smithi146.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-11T07:54:20.630 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-11T07:54:20.646 INFO:teuthology.orchestra.run.smithi146.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 543 kB/s | 59 kB 00:00 2024-09-11T07:54:20.680 INFO:teuthology.orchestra.run.smithi146.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 5.8 MB/s | 1.0 MB 00:00 2024-09-11T07:54:20.683 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:54:20.683 INFO:teuthology.orchestra.run.smithi146.stdout:Total 3.2 MB/s | 2.2 MB 00:00 2024-09-11T07:54:20.740 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-11T07:54:20.747 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-11T07:54:20.763 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-11T07:54:20.763 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-11T07:54:20.842 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-11T07:54:20.854 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-11T07:54:20.856 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-11T07:54:20.906 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-11T07:54:21.365 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-11T07:54:21.536 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-11T07:54:21.598 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-11T07:54:21.619 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-11T07:54:21.661 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-11T07:54:21.679 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-11T07:54:21.699 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-11T07:54:21.713 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-11T07:54:21.740 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-11T07:54:21.759 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-11T07:54:21.791 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T07:54:21.850 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-11T07:54:21.951 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-11T07:54:22.009 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-11T07:54:22.613 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T07:54:22.613 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 1/13 2024-09-11T07:54:22.613 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-11T07:54:22.614 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-11T07:54:22.614 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-11T07:54:22.614 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-11T07:54:22.614 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-11T07:54:22.614 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-11T07:54:22.615 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-11T07:54:22.615 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-11T07:54:22.615 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-11T07:54:22.615 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-11T07:54:22.615 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-11T07:54:22.732 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-11T07:54:22.781 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-11T07:54:22.814 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-11T07:54:22.860 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-11T07:54:22.884 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T07:54:22.942 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-11T07:54:22.942 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:22.942 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-11T07:54:22.943 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:54:22.943 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T07:54:22.943 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T07:54:22.943 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T07:54:22.943 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T07:54:22.943 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T07:54:22.943 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T07:54:22.943 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T07:54:22.943 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T07:54:22.943 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T07:54:22.943 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T07:54:22.943 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T07:54:22.943 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T07:54:22.943 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:22.944 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:54:23.235 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mgr-cephadm 2024-09-11T07:54:23.691 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 13/13 2024-09-11T07:54:23.692 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 1/13 2024-09-11T07:54:23.692 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-11T07:54:23.692 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-11T07:54:23.692 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-11T07:54:23.692 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-11T07:54:23.693 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-11T07:54:23.693 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-11T07:54:23.693 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-11T07:54:23.693 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-11T07:54:23.693 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-11T07:54:23.693 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-11T07:54:23.694 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-11T07:54:23.802 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:21 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:54:23.936 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:54:23.936 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:23.937 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-11T07:54:23.937 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:23.937 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-11T07:54:23.937 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 124 k 2024-09-11T07:54:23.937 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-11T07:54:23.937 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T07:54:23.937 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T07:54:23.937 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T07:54:23.937 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T07:54:23.937 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T07:54:23.937 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:23.937 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-11T07:54:23.938 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:23.938 INFO:teuthology.orchestra.run.smithi102.stdout:Install 6 Packages 2024-09-11T07:54:23.938 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:23.938 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 7.0 M 2024-09-11T07:54:23.938 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 33 M 2024-09-11T07:54:23.938 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-11T07:54:24.059 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-11T07:54:24.059 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:24.060 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-11T07:54:24.060 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:54:24.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T07:54:24.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T07:54:24.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T07:54:24.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T07:54:24.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T07:54:24.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T07:54:24.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T07:54:24.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T07:54:24.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T07:54:24.060 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T07:54:24.061 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T07:54:24.061 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T07:54:24.061 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:24.061 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:54:24.333 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-mgr-cephadm 2024-09-11T07:54:24.464 INFO:teuthology.orchestra.run.smithi102.stdout:(1/6): ceph-mgr-cephadm-17.2.7-1623.gd0e6828a.e 570 kB/s | 124 kB 00:00 2024-09-11T07:54:24.489 INFO:teuthology.orchestra.run.smithi102.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.0 MB/s | 248 kB 00:00 2024-09-11T07:54:24.539 INFO:teuthology.orchestra.run.smithi102.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-11T07:54:24.565 INFO:teuthology.orchestra.run.smithi102.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 766 kB/s | 58 kB 00:00 2024-09-11T07:54:24.590 INFO:teuthology.orchestra.run.smithi102.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.5 MB/s | 74 kB 00:00 2024-09-11T07:54:24.765 INFO:teuthology.orchestra.run.smithi102.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-09-11T07:54:24.767 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:54:24.767 INFO:teuthology.orchestra.run.smithi102.stdout:Total 8.4 MB/s | 7.0 MB 00:00 2024-09-11T07:54:24.839 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-11T07:54:24.849 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-11T07:54:24.849 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-11T07:54:24.896 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:01:21 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:54:24.950 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-11T07:54:24.950 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-11T07:54:25.033 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:54:25.033 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:25.033 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-09-11T07:54:25.033 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:25.033 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-11T07:54:25.034 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 124 k 2024-09-11T07:54:25.034 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-09-11T07:54:25.034 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T07:54:25.034 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T07:54:25.034 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T07:54:25.034 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T07:54:25.034 INFO:teuthology.orchestra.run.smithi146.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T07:54:25.034 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:25.034 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-11T07:54:25.034 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:25.034 INFO:teuthology.orchestra.run.smithi146.stdout:Install 6 Packages 2024-09-11T07:54:25.034 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:25.035 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 7.0 M 2024-09-11T07:54:25.035 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 33 M 2024-09-11T07:54:25.035 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-11T07:54:25.253 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-11T07:54:25.290 INFO:teuthology.orchestra.run.smithi146.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.6 MB/s | 248 kB 00:00 2024-09-11T07:54:25.349 INFO:teuthology.orchestra.run.smithi146.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-11T07:54:25.373 INFO:teuthology.orchestra.run.smithi146.stdout:(3/6): ceph-mgr-cephadm-17.2.7-1623.gd0e6828a.e 530 kB/s | 124 kB 00:00 2024-09-11T07:54:25.390 INFO:teuthology.orchestra.run.smithi146.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.4 MB/s | 58 kB 00:00 2024-09-11T07:54:25.431 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-11T07:54:25.432 INFO:teuthology.orchestra.run.smithi146.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.2 MB/s | 74 kB 00:00 2024-09-11T07:54:25.484 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-11T07:54:25.508 INFO:teuthology.orchestra.run.smithi146.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-11T07:54:25.510 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:54:25.510 INFO:teuthology.orchestra.run.smithi146.stdout:Total 15 MB/s | 7.0 MB 00:00 2024-09-11T07:54:25.580 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-11T07:54:25.592 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-11T07:54:25.592 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-11T07:54:25.690 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-11T07:54:25.691 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-11T07:54:26.001 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-11T07:54:26.006 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-11T07:54:26.095 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-11T07:54:26.158 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-11T07:54:26.184 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-11T07:54:26.187 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T07:54:26.234 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-11T07:54:26.767 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-11T07:54:26.863 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-11T07:54:26.918 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-11T07:54:26.955 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T07:54:26.989 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T07:54:26.989 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/6 2024-09-11T07:54:26.989 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-11T07:54:26.989 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-11T07:54:26.990 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-11T07:54:26.990 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-11T07:54:27.256 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-11T07:54:27.256 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:27.256 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-11T07:54:27.257 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:54:27.257 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T07:54:27.257 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T07:54:27.257 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T07:54:27.257 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T07:54:27.257 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T07:54:27.257 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:27.257 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:54:27.525 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-fuse 2024-09-11T07:54:27.748 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 6/6 2024-09-11T07:54:27.749 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 1/6 2024-09-11T07:54:27.749 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-11T07:54:27.749 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-11T07:54:27.749 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-11T07:54:27.749 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-11T07:54:28.033 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-11T07:54:28.034 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:28.034 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-11T07:54:28.034 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:54:28.034 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T07:54:28.034 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T07:54:28.034 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T07:54:28.034 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T07:54:28.034 INFO:teuthology.orchestra.run.smithi146.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T07:54:28.034 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:28.034 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:54:28.085 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:26 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:54:28.217 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:54:28.217 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:28.218 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-11T07:54:28.218 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:28.218 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-11T07:54:28.218 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 813 k 2024-09-11T07:54:28.218 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-11T07:54:28.218 INFO:teuthology.orchestra.run.smithi102.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T07:54:28.218 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:28.218 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-11T07:54:28.218 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:28.218 INFO:teuthology.orchestra.run.smithi102.stdout:Install 2 Packages 2024-09-11T07:54:28.218 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:28.219 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 893 k 2024-09-11T07:54:28.219 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 2.6 M 2024-09-11T07:54:28.219 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-11T07:54:28.299 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-fuse 2024-09-11T07:54:28.626 INFO:teuthology.orchestra.run.smithi102.stdout:(1/2): ceph-fuse-17.2.7-1623.gd0e6828a.el9.x86_ 2.8 MB/s | 813 kB 00:00 2024-09-11T07:54:28.859 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:01:25 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:54:28.991 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:54:28.992 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:28.992 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-09-11T07:54:28.992 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:28.992 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-11T07:54:28.992 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 813 k 2024-09-11T07:54:28.992 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-09-11T07:54:28.992 INFO:teuthology.orchestra.run.smithi146.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T07:54:28.992 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:28.993 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-11T07:54:28.993 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:28.993 INFO:teuthology.orchestra.run.smithi146.stdout:Install 2 Packages 2024-09-11T07:54:28.993 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:28.993 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 893 k 2024-09-11T07:54:28.993 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 2.6 M 2024-09-11T07:54:28.994 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-11T07:54:29.051 INFO:teuthology.orchestra.run.smithi102.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 112 kB/s | 80 kB 00:00 2024-09-11T07:54:29.051 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:54:29.052 INFO:teuthology.orchestra.run.smithi102.stdout:Total 1.0 MB/s | 893 kB 00:00 2024-09-11T07:54:29.064 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-11T07:54:29.088 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-11T07:54:29.088 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-11T07:54:29.174 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-11T07:54:29.174 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-11T07:54:29.300 INFO:teuthology.orchestra.run.smithi146.stdout:(1/2): ceph-fuse-17.2.7-1623.gd0e6828a.el9.x86_ 3.1 MB/s | 813 kB 00:00 2024-09-11T07:54:29.366 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-11T07:54:29.459 INFO:teuthology.orchestra.run.smithi146.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 190 kB/s | 80 kB 00:00 2024-09-11T07:54:29.459 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:54:29.460 INFO:teuthology.orchestra.run.smithi146.stdout:Total 1.9 MB/s | 893 kB 00:00 2024-09-11T07:54:29.471 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-11T07:54:29.493 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-11T07:54:29.493 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-11T07:54:29.529 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-11T07:54:29.575 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-11T07:54:29.575 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-11T07:54:29.599 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T07:54:29.765 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-11T07:54:29.905 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-11T07:54:29.975 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T07:54:30.248 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T07:54:30.248 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/2 2024-09-11T07:54:30.518 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-11T07:54:30.518 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:30.518 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-11T07:54:30.518 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-11T07:54:30.518 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:30.518 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:54:30.630 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/2 2024-09-11T07:54:30.630 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/2 2024-09-11T07:54:30.832 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-volume 2024-09-11T07:54:30.918 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-11T07:54:30.918 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:30.918 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-11T07:54:30.918 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-11T07:54:30.918 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:30.918 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:54:31.238 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-volume 2024-09-11T07:54:31.413 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:29 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:54:31.546 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:54:31.547 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:31.547 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-11T07:54:31.547 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:31.547 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-11T07:54:31.547 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 270 k 2024-09-11T07:54:31.548 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:31.548 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-11T07:54:31.548 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:31.548 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-11T07:54:31.548 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:31.548 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 270 k 2024-09-11T07:54:31.548 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 1.3 M 2024-09-11T07:54:31.548 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-11T07:54:31.756 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-volume-17.2.7-1623.gd0e6828a.el9.noarch.rp 1.3 MB/s | 270 kB 00:00 2024-09-11T07:54:31.758 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:54:31.758 INFO:teuthology.orchestra.run.smithi102.stdout:Total 1.3 MB/s | 270 kB 00:00 2024-09-11T07:54:31.758 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-11T07:54:31.765 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-11T07:54:31.765 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-11T07:54:31.797 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-11T07:54:31.798 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-11T07:54:31.892 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:01:28 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:54:32.006 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-11T07:54:32.028 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:54:32.029 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:32.029 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-09-11T07:54:32.029 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:32.029 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-11T07:54:32.029 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 270 k 2024-09-11T07:54:32.029 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:32.029 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-11T07:54:32.030 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:32.030 INFO:teuthology.orchestra.run.smithi146.stdout:Install 1 Package 2024-09-11T07:54:32.030 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:32.030 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 270 k 2024-09-11T07:54:32.030 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 1.3 M 2024-09-11T07:54:32.030 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-11T07:54:32.075 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:54:32.091 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:54:32.091 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:54:32.091 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T07:54:32.091 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:32.238 INFO:teuthology.orchestra.run.smithi146.stdout:ceph-volume-17.2.7-1623.gd0e6828a.el9.noarch.rp 1.3 MB/s | 270 kB 00:00 2024-09-11T07:54:32.239 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:54:32.239 INFO:teuthology.orchestra.run.smithi146.stdout:Total 1.3 MB/s | 270 kB 00:00 2024-09-11T07:54:32.240 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-11T07:54:32.248 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-11T07:54:32.248 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-11T07:54:32.274 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-11T07:54:32.275 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-11T07:54:32.490 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-11T07:54:32.557 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:54:32.573 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:54:32.574 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:54:32.574 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T07:54:32.574 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:33.081 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:54:33.081 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:33.082 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-11T07:54:33.082 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:54:33.082 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:33.082 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:54:33.399 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install librados-devel 2024-09-11T07:54:33.668 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 1/1 2024-09-11T07:54:33.668 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:33.668 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-11T07:54:33.669 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:54:33.669 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:33.669 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:54:33.976 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install librados-devel 2024-09-11T07:54:33.981 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:31 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:54:34.119 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:54:34.120 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:34.120 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-11T07:54:34.120 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:34.120 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-11T07:54:34.120 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 126 k 2024-09-11T07:54:34.120 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:34.120 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-11T07:54:34.120 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:34.120 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-11T07:54:34.120 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:34.121 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 126 k 2024-09-11T07:54:34.121 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 448 k 2024-09-11T07:54:34.121 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-11T07:54:34.336 INFO:teuthology.orchestra.run.smithi102.stdout:librados-devel-17.2.7-1623.gd0e6828a.el9.x86_64 587 kB/s | 126 kB 00:00 2024-09-11T07:54:34.337 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:54:34.337 INFO:teuthology.orchestra.run.smithi102.stdout:Total 582 kB/s | 126 kB 00:00 2024-09-11T07:54:34.337 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-11T07:54:34.343 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-11T07:54:34.343 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-11T07:54:34.407 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-11T07:54:34.407 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-11T07:54:34.552 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-11T07:54:34.558 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:01:31 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:54:34.642 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:34.694 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:54:34.694 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:34.694 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repo Size 2024-09-11T07:54:34.694 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:34.695 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-11T07:54:34.695 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 126 k 2024-09-11T07:54:34.695 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:34.695 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-11T07:54:34.695 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:34.695 INFO:teuthology.orchestra.run.smithi146.stdout:Install 1 Package 2024-09-11T07:54:34.695 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:34.695 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 126 k 2024-09-11T07:54:34.695 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 448 k 2024-09-11T07:54:34.695 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-11T07:54:34.901 INFO:teuthology.orchestra.run.smithi146.stdout:librados-devel-17.2.7-1623.gd0e6828a.el9.x86_64 614 kB/s | 126 kB 00:00 2024-09-11T07:54:34.902 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:54:34.902 INFO:teuthology.orchestra.run.smithi146.stdout:Total 610 kB/s | 126 kB 00:00 2024-09-11T07:54:34.902 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-11T07:54:34.907 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-11T07:54:34.907 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-11T07:54:34.971 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-11T07:54:34.972 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-11T07:54:35.057 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:35.118 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-11T07:54:35.222 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:35.306 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:35.307 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:35.307 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-11T07:54:35.307 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:54:35.307 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:35.307 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:54:35.622 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install libcephfs2 2024-09-11T07:54:35.710 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:35.975 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:35.975 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:35.976 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-11T07:54:35.976 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:54:35.976 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:35.976 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:54:36.198 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:34 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:54:36.290 INFO:teuthology.orchestra.run.smithi102.stdout:Package libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:54:36.295 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install libcephfs2 2024-09-11T07:54:36.334 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:54:36.335 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-11T07:54:36.335 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:54:36.403 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install libcephfs-devel 2024-09-11T07:54:36.863 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:01:33 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:54:36.955 INFO:teuthology.orchestra.run.smithi146.stdout:Package libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:54:36.966 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:34 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:54:36.999 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:54:37.000 INFO:teuthology.orchestra.run.smithi146.stdout:Nothing to do. 2024-09-11T07:54:37.001 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:54:37.075 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install libcephfs-devel 2024-09-11T07:54:37.098 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:54:37.099 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:37.099 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-11T07:54:37.100 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:37.100 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-11T07:54:37.100 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T07:54:37.100 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:37.100 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-11T07:54:37.100 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:37.100 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-11T07:54:37.100 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:37.100 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 25 k 2024-09-11T07:54:37.100 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 110 k 2024-09-11T07:54:37.100 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-11T07:54:37.241 INFO:teuthology.orchestra.run.smithi102.stdout:libcephfs-devel-17.2.7-1623.gd0e6828a.el9.x86_6 177 kB/s | 25 kB 00:00 2024-09-11T07:54:37.241 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:54:37.241 INFO:teuthology.orchestra.run.smithi102.stdout:Total 176 kB/s | 25 kB 00:00 2024-09-11T07:54:37.242 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-11T07:54:37.245 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-11T07:54:37.245 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-11T07:54:37.246 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-11T07:54:37.247 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-11T07:54:37.322 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-11T07:54:37.428 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:37.638 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:01:34 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:54:37.773 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:54:37.774 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:37.774 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repo Size 2024-09-11T07:54:37.774 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:37.774 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-11T07:54:37.774 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T07:54:37.774 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:37.775 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-11T07:54:37.775 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:37.775 INFO:teuthology.orchestra.run.smithi146.stdout:Install 1 Package 2024-09-11T07:54:37.775 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:37.775 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 25 k 2024-09-11T07:54:37.775 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 110 k 2024-09-11T07:54:37.775 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-11T07:54:37.810 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:37.930 INFO:teuthology.orchestra.run.smithi146.stdout:libcephfs-devel-17.2.7-1623.gd0e6828a.el9.x86_6 160 kB/s | 25 kB 00:00 2024-09-11T07:54:37.931 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:54:37.931 INFO:teuthology.orchestra.run.smithi146.stdout:Total 159 kB/s | 25 kB 00:00 2024-09-11T07:54:37.931 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-11T07:54:37.934 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-11T07:54:37.934 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-11T07:54:37.936 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-11T07:54:37.936 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-11T07:54:38.021 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-11T07:54:38.055 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:38.056 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:38.056 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-11T07:54:38.056 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:54:38.056 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:38.056 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:54:38.126 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:38.219 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install librados2 2024-09-11T07:54:38.517 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:38.773 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:38.774 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:38.774 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-11T07:54:38.774 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:54:38.774 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:38.774 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:54:38.780 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:36 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:54:38.869 INFO:teuthology.orchestra.run.smithi102.stdout:Package librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:54:38.912 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:54:38.913 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-11T07:54:38.913 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:54:38.970 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install librados2 2024-09-11T07:54:38.986 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install librbd1 2024-09-11T07:54:39.537 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:01:36 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:54:39.544 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:37 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:54:39.629 INFO:teuthology.orchestra.run.smithi146.stdout:Package librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:54:39.633 INFO:teuthology.orchestra.run.smithi102.stdout:Package librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:54:39.672 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:54:39.673 INFO:teuthology.orchestra.run.smithi146.stdout:Nothing to do. 2024-09-11T07:54:39.673 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:54:39.676 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:54:39.677 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-11T07:54:39.677 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:54:39.743 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-rados 2024-09-11T07:54:39.763 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install librbd1 2024-09-11T07:54:40.303 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:38 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:54:40.326 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:01:37 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:54:40.392 INFO:teuthology.orchestra.run.smithi102.stdout:Package python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:54:40.421 INFO:teuthology.orchestra.run.smithi146.stdout:Package librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:54:40.434 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:54:40.435 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-11T07:54:40.435 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:54:40.460 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:54:40.461 INFO:teuthology.orchestra.run.smithi146.stdout:Nothing to do. 2024-09-11T07:54:40.461 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:54:40.500 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-rgw 2024-09-11T07:54:40.547 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install python3-rados 2024-09-11T07:54:41.055 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:39 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:54:41.108 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:01:38 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:54:41.144 INFO:teuthology.orchestra.run.smithi102.stdout:Package python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:54:41.187 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:54:41.188 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-11T07:54:41.188 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:54:41.197 INFO:teuthology.orchestra.run.smithi146.stdout:Package python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:54:41.241 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:54:41.241 INFO:teuthology.orchestra.run.smithi146.stdout:Nothing to do. 2024-09-11T07:54:41.241 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:54:41.261 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-cephfs 2024-09-11T07:54:41.318 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install python3-rgw 2024-09-11T07:54:41.824 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:39 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:54:41.882 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:01:38 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:54:41.915 INFO:teuthology.orchestra.run.smithi102.stdout:Package python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:54:41.957 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:54:41.958 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-11T07:54:41.958 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:54:41.971 INFO:teuthology.orchestra.run.smithi146.stdout:Package python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:54:42.015 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:54:42.016 INFO:teuthology.orchestra.run.smithi146.stdout:Nothing to do. 2024-09-11T07:54:42.016 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:54:42.028 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-rbd 2024-09-11T07:54:42.101 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install python3-cephfs 2024-09-11T07:54:42.595 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:40 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:54:42.659 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:01:39 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:54:42.684 INFO:teuthology.orchestra.run.smithi102.stdout:Package python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:54:42.726 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:54:42.727 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-11T07:54:42.727 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:54:42.750 INFO:teuthology.orchestra.run.smithi146.stdout:Package python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:54:42.793 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:54:42.794 INFO:teuthology.orchestra.run.smithi146.stdout:Nothing to do. 2024-09-11T07:54:42.794 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:54:42.799 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install rbd-fuse 2024-09-11T07:54:42.877 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install python3-rbd 2024-09-11T07:54:43.360 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:41 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:54:43.439 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:01:40 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:54:43.494 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:54:43.494 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:43.495 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-09-11T07:54:43.495 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:43.495 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-11T07:54:43.495 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 82 k 2024-09-11T07:54:43.495 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:43.495 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-11T07:54:43.495 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:43.495 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-11T07:54:43.495 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:43.495 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 82 k 2024-09-11T07:54:43.495 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 222 k 2024-09-11T07:54:43.496 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-11T07:54:43.529 INFO:teuthology.orchestra.run.smithi146.stdout:Package python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 is already installed. 2024-09-11T07:54:43.571 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:54:43.572 INFO:teuthology.orchestra.run.smithi146.stdout:Nothing to do. 2024-09-11T07:54:43.572 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:54:43.655 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install rbd-fuse 2024-09-11T07:54:43.686 INFO:teuthology.orchestra.run.smithi102.stdout:rbd-fuse-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 431 kB/s | 82 kB 00:00 2024-09-11T07:54:43.686 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:54:43.686 INFO:teuthology.orchestra.run.smithi102.stdout:Total 428 kB/s | 82 kB 00:00 2024-09-11T07:54:43.686 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-11T07:54:43.697 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-11T07:54:43.697 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-11T07:54:43.757 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-11T07:54:43.758 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-11T07:54:43.897 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-11T07:54:44.008 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:44.212 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:01:41 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:54:44.345 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:54:44.346 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:44.346 INFO:teuthology.orchestra.run.smithi146.stdout: Package Architecture Version Repository Size 2024-09-11T07:54:44.346 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:44.346 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-11T07:54:44.346 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 82 k 2024-09-11T07:54:44.346 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:44.346 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-11T07:54:44.346 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:44.346 INFO:teuthology.orchestra.run.smithi146.stdout:Install 1 Package 2024-09-11T07:54:44.347 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:44.347 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 82 k 2024-09-11T07:54:44.347 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 222 k 2024-09-11T07:54:44.347 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-11T07:54:44.374 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:44.550 INFO:teuthology.orchestra.run.smithi146.stdout:rbd-fuse-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 404 kB/s | 82 kB 00:00 2024-09-11T07:54:44.550 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:54:44.550 INFO:teuthology.orchestra.run.smithi146.stdout:Total 401 kB/s | 82 kB 00:00 2024-09-11T07:54:44.551 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-11T07:54:44.557 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-11T07:54:44.557 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-11T07:54:44.621 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-11T07:54:44.621 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-11T07:54:44.653 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:44.653 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:44.654 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-11T07:54:44.654 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:54:44.654 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:44.654 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:54:44.762 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-11T07:54:44.862 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:44.877 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install rbd-mirror 2024-09-11T07:54:45.283 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:45.451 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:43 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:54:45.540 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:45.540 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:45.541 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-11T07:54:45.541 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:54:45.541 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:45.541 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:54:45.585 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:54:45.586 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:45.586 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-11T07:54:45.586 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:45.586 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-11T07:54:45.587 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T07:54:45.587 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:45.587 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-11T07:54:45.587 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:45.587 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-11T07:54:45.587 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:45.587 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 2.9 M 2024-09-11T07:54:45.587 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 12 M 2024-09-11T07:54:45.587 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-11T07:54:45.791 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install rbd-mirror 2024-09-11T07:54:45.926 INFO:teuthology.orchestra.run.smithi102.stdout:rbd-mirror-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 8.7 MB/s | 2.9 MB 00:00 2024-09-11T07:54:45.926 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:54:45.926 INFO:teuthology.orchestra.run.smithi102.stdout:Total 8.6 MB/s | 2.9 MB 00:00 2024-09-11T07:54:45.927 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-11T07:54:45.935 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-11T07:54:45.935 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-11T07:54:46.016 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-11T07:54:46.016 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-11T07:54:46.373 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:01:43 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:54:46.432 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-11T07:54:46.490 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:46.509 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:54:46.510 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:46.510 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repo Size 2024-09-11T07:54:46.511 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:46.511 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-11T07:54:46.511 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T07:54:46.511 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:46.511 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-11T07:54:46.511 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:46.511 INFO:teuthology.orchestra.run.smithi146.stdout:Install 1 Package 2024-09-11T07:54:46.512 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:46.512 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 2.9 M 2024-09-11T07:54:46.512 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 12 M 2024-09-11T07:54:46.512 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-11T07:54:46.514 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:46.515 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:54:46.515 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T07:54:46.515 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T07:54:46.515 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T07:54:46.515 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:46.844 INFO:teuthology.orchestra.run.smithi146.stdout:rbd-mirror-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 8.8 MB/s | 2.9 MB 00:00 2024-09-11T07:54:46.845 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:54:46.845 INFO:teuthology.orchestra.run.smithi146.stdout:Total 8.8 MB/s | 2.9 MB 00:00 2024-09-11T07:54:46.845 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-11T07:54:46.854 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-11T07:54:46.854 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-11T07:54:46.934 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-11T07:54:46.935 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-11T07:54:47.363 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-11T07:54:47.412 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:47.437 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:47.437 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:54:47.437 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T07:54:47.437 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T07:54:47.437 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T07:54:47.437 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:47.631 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:47.631 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:47.631 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-11T07:54:47.631 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:54:47.632 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:47.632 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:54:47.866 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install rbd-nbd 2024-09-11T07:54:48.441 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:46 ago on Wed 11 Sep 2024 07:53:02 AM UTC. 2024-09-11T07:54:48.533 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:48.533 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:48.533 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-11T07:54:48.533 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:54:48.533 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:48.534 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:54:48.580 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-11T07:54:48.581 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:48.581 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-09-11T07:54:48.581 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:48.581 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-11T07:54:48.581 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 165 k 2024-09-11T07:54:48.581 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:48.581 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-11T07:54:48.582 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-11T07:54:48.582 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-11T07:54:48.582 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:48.582 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 165 k 2024-09-11T07:54:48.582 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 477 k 2024-09-11T07:54:48.582 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-11T07:54:48.763 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install rbd-nbd 2024-09-11T07:54:48.802 INFO:teuthology.orchestra.run.smithi102.stdout:rbd-nbd-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 752 kB/s | 165 kB 00:00 2024-09-11T07:54:48.803 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:54:48.803 INFO:teuthology.orchestra.run.smithi102.stdout:Total 745 kB/s | 165 kB 00:00 2024-09-11T07:54:48.803 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-11T07:54:48.812 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-11T07:54:48.812 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-11T07:54:48.878 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-11T07:54:48.879 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-11T07:54:49.037 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-11T07:54:49.141 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:49.346 INFO:teuthology.orchestra.run.smithi146.stdout:Last metadata expiration check: 0:01:46 ago on Wed 11 Sep 2024 07:53:03 AM UTC. 2024-09-11T07:54:49.484 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-11T07:54:49.485 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:49.485 INFO:teuthology.orchestra.run.smithi146.stdout: Package Architecture Version Repository Size 2024-09-11T07:54:49.485 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:49.485 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-11T07:54:49.485 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 165 k 2024-09-11T07:54:49.485 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:49.485 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-11T07:54:49.485 INFO:teuthology.orchestra.run.smithi146.stdout:================================================================================ 2024-09-11T07:54:49.486 INFO:teuthology.orchestra.run.smithi146.stdout:Install 1 Package 2024-09-11T07:54:49.486 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:49.486 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 165 k 2024-09-11T07:54:49.486 INFO:teuthology.orchestra.run.smithi146.stdout:Installed size: 477 k 2024-09-11T07:54:49.486 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-11T07:54:49.613 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:49.681 INFO:teuthology.orchestra.run.smithi146.stdout:rbd-nbd-17.2.7-1623.gd0e6828a.el9.x86_64.rpm 844 kB/s | 165 kB 00:00 2024-09-11T07:54:49.682 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:54:49.682 INFO:teuthology.orchestra.run.smithi146.stdout:Total 836 kB/s | 165 kB 00:00 2024-09-11T07:54:49.682 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-11T07:54:49.690 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-11T07:54:49.690 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-11T07:54:49.758 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-11T07:54:49.758 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-11T07:54:49.881 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:49.881 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:49.881 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-11T07:54:49.881 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:54:49.881 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:54:49.881 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-11T07:54:49.917 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-11T07:54:50.013 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:50.123 DEBUG:teuthology.parallel:result is None 2024-09-11T07:54:50.474 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:50.749 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/1 2024-09-11T07:54:50.749 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:50.749 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-11T07:54:50.749 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:54:50.749 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:54:50.749 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-11T07:54:51.009 DEBUG:teuthology.parallel:result is None 2024-09-11T07:54:51.010 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:54:51.158 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T07:54:51.183 INFO:teuthology.orchestra.run.smithi102.stdout:17.2.7-1623.gd0e6828a.el9 2024-09-11T07:54:51.184 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1623.gd0e6828a.el9 2024-09-11T07:54:51.184 INFO:teuthology.task.install:The correct ceph version 17.2.7-1623.gd0e6828a is installed. 2024-09-11T07:54:51.186 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:54:51.328 DEBUG:teuthology.orchestra.run.smithi146:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T07:54:51.349 INFO:teuthology.orchestra.run.smithi146.stdout:17.2.7-1623.gd0e6828a.el9 2024-09-11T07:54:51.350 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1623.gd0e6828a.el9 2024-09-11T07:54:51.350 INFO:teuthology.task.install:The correct ceph version 17.2.7-1623.gd0e6828a is installed. 2024-09-11T07:54:51.351 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T07:54:51.352 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-11T07:54:51.352 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T07:54:51.383 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-11T07:54:51.383 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T07:54:51.417 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T07:54:51.417 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-11T07:54:51.417 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T07:54:51.452 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T07:54:51.519 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-11T07:54:51.519 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T07:54:51.543 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T07:54:51.608 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T07:54:51.608 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-11T07:54:51.608 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T07:54:51.637 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T07:54:51.704 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-11T07:54:51.705 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T07:54:51.729 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T07:54:51.794 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T07:54:51.794 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-11T07:54:51.794 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T07:54:51.821 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T07:54:51.887 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-11T07:54:51.888 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T07:54:51.912 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T07:54:51.977 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-11T07:54:52.067 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 1, 'ms bind msgr2': False, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'mon_bind_msgr2': False, 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'cephadm_mode': 'cephadm-package'} 2024-09-11T07:54:52.067 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:54:52.068 INFO:tasks.cephadm:Cluster fsid is 2051892c-7013-11ef-bcea-c7b262605968 2024-09-11T07:54:52.068 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-11T07:54:52.068 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '[v1:172.21.15.102:6789]', 'mon.c': '[v1:172.21.15.102:6790]', 'mon.b': '[v1:172.21.15.146:6789]'} 2024-09-11T07:54:52.068 INFO:tasks.cephadm:First mon is mon.a on smithi102 2024-09-11T07:54:52.068 INFO:tasks.cephadm:First mgr is y 2024-09-11T07:54:52.068 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-11T07:54:52.068 DEBUG:teuthology.orchestra.run.smithi102:> sudo hostname $(hostname -s) 2024-09-11T07:54:52.096 DEBUG:teuthology.orchestra.run.smithi146:> sudo hostname $(hostname -s) 2024-09-11T07:54:52.132 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 on all hosts... 2024-09-11T07:54:52.132 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 pull 2024-09-11T07:54:52.138 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 pull 2024-09-11T07:54:53.708 INFO:teuthology.orchestra.run.smithi102.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T07:54:53.827 INFO:teuthology.orchestra.run.smithi146.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T07:55:19.950 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-11T07:55:19.950 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T07:55:19.951 INFO:teuthology.orchestra.run.smithi146.stdout: "image_id": "4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b", 2024-09-11T07:55:19.951 INFO:teuthology.orchestra.run.smithi146.stdout: "repo_digests": [ 2024-09-11T07:55:19.951 INFO:teuthology.orchestra.run.smithi146.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40" 2024-09-11T07:55:19.951 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-09-11T07:55:19.951 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-11T07:55:21.822 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-11T07:55:21.823 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T07:55:21.823 INFO:teuthology.orchestra.run.smithi102.stdout: "image_id": "4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b", 2024-09-11T07:55:21.823 INFO:teuthology.orchestra.run.smithi102.stdout: "repo_digests": [ 2024-09-11T07:55:21.823 INFO:teuthology.orchestra.run.smithi102.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40" 2024-09-11T07:55:21.823 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-11T07:55:21.823 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-11T07:55:21.842 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /etc/ceph 2024-09-11T07:55:21.890 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph 2024-09-11T07:55:21.922 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 777 /etc/ceph 2024-09-11T07:55:21.947 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 777 /etc/ceph 2024-09-11T07:55:21.990 INFO:tasks.cephadm:Writing seed config... 2024-09-11T07:55:21.992 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-11T07:55:21.992 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-11T07:55:21.992 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-11T07:55:21.992 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-09-11T07:55:21.992 INFO:tasks.cephadm: override: [global] ms bind msgr2 = False 2024-09-11T07:55:21.992 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-11T07:55:21.992 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-11T07:55:21.992 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-11T07:55:21.992 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-11T07:55:21.992 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-11T07:55:21.995 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-11T07:55:21.996 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-11T07:55:21.996 INFO:tasks.cephadm: override: [osd] osd class default list = * 2024-09-11T07:55:21.996 INFO:tasks.cephadm: override: [osd] osd class load list = * 2024-09-11T07:55:21.996 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-11T07:55:21.996 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-11T07:55:21.997 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-11T07:55:22.013 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 2051892c-7013-11ef-bcea-c7b262605968 mon election default strategy = 1 ms bind msgr2 = False ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd class default list = * osd class load list = * [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-09-11T07:55:22.013 DEBUG:teuthology.orchestra.run.smithi102:mon.a> sudo journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@mon.a.service 2024-09-11T07:55:22.055 DEBUG:teuthology.orchestra.run.smithi102:mgr.y> sudo journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@mgr.y.service 2024-09-11T07:55:22.098 INFO:tasks.cephadm:Bootstrapping... 2024-09-11T07:55:22.099 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 -v bootstrap --fsid 2051892c-7013-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-addrv '[v1:172.21.15.102:6789]' --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:55:22.296 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:55:22.296 INFO:teuthology.orchestra.run.smithi102.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9', '-v', 'bootstrap', '--fsid', '2051892c-7013-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-addrv', '[v1:172.21.15.102:6789]', '--skip-admin-label'] 2024-09-11T07:55:22.320 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stdout 5.2.2 2024-09-11T07:55:22.320 INFO:teuthology.orchestra.run.smithi102.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-11T07:55:22.320 INFO:teuthology.orchestra.run.smithi102.stdout:Verifying podman|docker is present... 2024-09-11T07:55:22.342 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stdout 5.2.2 2024-09-11T07:55:22.342 INFO:teuthology.orchestra.run.smithi102.stdout:Verifying lvm2 is present... 2024-09-11T07:55:22.342 INFO:teuthology.orchestra.run.smithi102.stdout:Verifying time synchronization is in place... 2024-09-11T07:55:22.350 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T07:55:22.350 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T07:55:22.357 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T07:55:22.357 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout inactive 2024-09-11T07:55:22.364 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout enabled 2024-09-11T07:55:22.370 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout active 2024-09-11T07:55:22.370 INFO:teuthology.orchestra.run.smithi102.stdout:Unit chronyd.service is enabled and running 2024-09-11T07:55:22.371 INFO:teuthology.orchestra.run.smithi102.stdout:Repeating the final host check... 2024-09-11T07:55:22.393 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stdout 5.2.2 2024-09-11T07:55:22.393 INFO:teuthology.orchestra.run.smithi102.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-11T07:55:22.393 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl is present 2024-09-11T07:55:22.393 INFO:teuthology.orchestra.run.smithi102.stdout:lvcreate is present 2024-09-11T07:55:22.399 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T07:55:22.400 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T07:55:22.406 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T07:55:22.406 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout inactive 2024-09-11T07:55:22.413 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout enabled 2024-09-11T07:55:22.420 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout active 2024-09-11T07:55:22.420 INFO:teuthology.orchestra.run.smithi102.stdout:Unit chronyd.service is enabled and running 2024-09-11T07:55:22.420 INFO:teuthology.orchestra.run.smithi102.stdout:Host looks OK 2024-09-11T07:55:22.420 INFO:teuthology.orchestra.run.smithi102.stdout:Cluster fsid: 2051892c-7013-11ef-bcea-c7b262605968 2024-09-11T07:55:22.420 INFO:teuthology.orchestra.run.smithi102.stdout:Acquiring lock 139677214167824 on /run/cephadm/2051892c-7013-11ef-bcea-c7b262605968.lock 2024-09-11T07:55:22.420 INFO:teuthology.orchestra.run.smithi102.stdout:Lock 139677214167824 acquired on /run/cephadm/2051892c-7013-11ef-bcea-c7b262605968.lock 2024-09-11T07:55:22.421 INFO:teuthology.orchestra.run.smithi102.stdout:Verifying IP 172.21.15.102 port 6789 ... 2024-09-11T07:55:22.421 INFO:teuthology.orchestra.run.smithi102.stdout:Base mon IP(s) is [172.21.15.102:6789], mon addrv is [v1:172.21.15.102:6789] 2024-09-11T07:55:22.424 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.102 metric 100 2024-09-11T07:55:22.424 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.102 metric 100 2024-09-11T07:55:22.427 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-11T07:55:22.427 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-11T07:55:22.427 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-11T07:55:22.430 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-11T07:55:22.430 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-11T07:55:22.430 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T07:55:22.430 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-11T07:55:22.430 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d769/64 scope link noprefixroute 2024-09-11T07:55:22.430 INFO:teuthology.orchestra.run.smithi102.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T07:55:22.430 INFO:teuthology.orchestra.run.smithi102.stdout:Mon IP `172.21.15.102` is in CIDR network `172.21.0.0/20` 2024-09-11T07:55:22.431 INFO:teuthology.orchestra.run.smithi102.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20'] 2024-09-11T07:55:22.431 INFO:teuthology.orchestra.run.smithi102.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-11T07:55:22.431 INFO:teuthology.orchestra.run.smithi102.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T07:55:22.889 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stdout 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b 2024-09-11T07:55:22.889 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T07:55:22.889 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stderr Getting image source signatures 2024-09-11T07:55:22.889 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stderr Copying blob sha256:0d2f264d63cec2f6fb0f85eed7b2c8c5b4db3ab9edd9d71a74c04647294a553f 2024-09-11T07:55:22.889 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-11T07:55:22.889 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stderr Copying config sha256:4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b 2024-09-11T07:55:22.889 INFO:teuthology.orchestra.run.smithi102.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-11T07:55:23.894 INFO:teuthology.orchestra.run.smithi102.stdout:ceph: stdout ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable) 2024-09-11T07:55:23.894 INFO:teuthology.orchestra.run.smithi102.stdout:Ceph version: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable) 2024-09-11T07:55:23.894 INFO:teuthology.orchestra.run.smithi102.stdout:Extracting ceph user uid/gid from container image... 2024-09-11T07:55:24.967 INFO:teuthology.orchestra.run.smithi102.stdout:stat: stdout 167 167 2024-09-11T07:55:24.967 INFO:teuthology.orchestra.run.smithi102.stdout:Creating initial keys... 2024-09-11T07:55:25.930 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph-authtool: stdout AQDtTOFmene5ERAAscpV6KVeq6tHZt84ljkkmw== 2024-09-11T07:55:26.877 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph-authtool: stdout AQDuTOFmttoWEhAAVhPQPrKtqK8zdAGUSNP6Cw== 2024-09-11T07:55:27.849 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph-authtool: stdout AQDvTOFmd073DRAA80/VnxO8MDEV+SOLzAkv9g== 2024-09-11T07:55:27.850 INFO:teuthology.orchestra.run.smithi102.stdout:Creating initial monmap... 2024-09-11T07:55:28.897 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T07:55:28.898 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-09-11T07:55:28.898 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 2051892c-7013-11ef-bcea-c7b262605968 2024-09-11T07:55:28.898 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T07:55:28.898 INFO:teuthology.orchestra.run.smithi102.stdout:monmaptool for a [v1:172.21.15.102:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T07:55:28.898 INFO:teuthology.orchestra.run.smithi102.stdout:setting min_mon_release = octopus 2024-09-11T07:55:28.898 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/monmaptool: set fsid to 2051892c-7013-11ef-bcea-c7b262605968 2024-09-11T07:55:28.898 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T07:55:28.898 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:55:28.898 INFO:teuthology.orchestra.run.smithi102.stdout:Creating mon... 2024-09-11T07:55:29.912 INFO:teuthology.orchestra.run.smithi102.stdout:create mon.a on 2024-09-11T07:55:30.116 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-11T07:55:30.265 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-11T07:55:30.419 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-2051892c-7013-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-2051892c-7013-11ef-bcea-c7b262605968.target. 2024-09-11T07:55:30.419 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-2051892c-7013-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-2051892c-7013-11ef-bcea-c7b262605968.target. 2024-09-11T07:55:30.588 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2051892c-7013-11ef-bcea-c7b262605968@mon.a 2024-09-11T07:55:30.589 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2051892c-7013-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-2051892c-7013-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-11T07:55:30.755 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2051892c-7013-11ef-bcea-c7b262605968.target.wants/ceph-2051892c-7013-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-2051892c-7013-11ef-bcea-c7b262605968@.service. 2024-09-11T07:55:31.244 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:31 smithi102 podman[29341]: 2024-09-11 07:55:31.209031754 +0000 UTC m=+0.337634684 container start 3c2d329aca3d833b0d8a3749f884c3ab1c2cbc8194c34401a52faedf302f5cf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-11T07:55:31.326 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T07:55:31.327 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout disabled 2024-09-11T07:55:31.334 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T07:55:31.334 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout inactive 2024-09-11T07:55:31.334 INFO:teuthology.orchestra.run.smithi102.stdout:firewalld.service is not enabled 2024-09-11T07:55:31.334 INFO:teuthology.orchestra.run.smithi102.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T07:55:31.334 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for mon to start... 2024-09-11T07:55:31.334 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for mon... 2024-09-11T07:55:31.550 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:31 smithi102 ceph-mon[29379]: mkfs 2051892c-7013-11ef-bcea-c7b262605968 2024-09-11T07:55:31.550 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:31 smithi102 ceph-mon[29379]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T07:55:31.550 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:31 smithi102 bash[29341]: 3c2d329aca3d833b0d8a3749f884c3ab1c2cbc8194c34401a52faedf302f5cf9 2024-09-11T07:55:31.550 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:31 smithi102 systemd[1]: Started Ceph mon.a for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T07:55:32.337 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout cluster: 2024-09-11T07:55:32.338 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout id: 2051892c-7013-11ef-bcea-c7b262605968 2024-09-11T07:55:32.338 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-11T07:55:32.338 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-09-11T07:55:32.338 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout services: 2024-09-11T07:55:32.338 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.564303s) 2024-09-11T07:55:32.338 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-11T07:55:32.338 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-11T07:55:32.338 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-09-11T07:55:32.338 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout data: 2024-09-11T07:55:32.338 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-11T07:55:32.338 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-11T07:55:32.339 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-11T07:55:32.339 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout pgs: 2024-09-11T07:55:32.339 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-09-11T07:55:32.339 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.740+0000 7fe71d337640 1 Processor -- start 2024-09-11T07:55:32.339 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.740+0000 7fe71d337640 1 -- start start 2024-09-11T07:55:32.339 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.740+0000 7fe71d337640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe718075250 con 0x7fe718076610 2024-09-11T07:55:32.339 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.740+0000 7fe716ffd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fe718076610 0x7fe718074b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:48846/0 (socket says 172.21.15.102:48846) 2024-09-11T07:55:32.339 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.740+0000 7fe716ffd640 1 -- 172.21.15.102:0/2004340720 learned_addr learned my addr 172.21.15.102:0/2004340720 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:55:32.339 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.741+0000 7fe715ffb640 1 -- 172.21.15.102:0/2004340720 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1642416781 0 0) 0x7fe718075250 con 0x7fe718076610 2024-09-11T07:55:32.339 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.741+0000 7fe715ffb640 1 -- 172.21.15.102:0/2004340720 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7000034e0 con 0x7fe718076610 2024-09-11T07:55:32.339 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.741+0000 7fe715ffb640 1 -- 172.21.15.102:0/2004340720 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3511233149 0 0) 0x7fe7000034e0 con 0x7fe718076610 2024-09-11T07:55:32.340 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.741+0000 7fe715ffb640 1 -- 172.21.15.102:0/2004340720 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7180753e0 con 0x7fe718076610 2024-09-11T07:55:32.340 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.741+0000 7fe715ffb640 1 -- 172.21.15.102:0/2004340720 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fe708001590 con 0x7fe718076610 2024-09-11T07:55:32.340 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.741+0000 7fe715ffb640 1 -- 172.21.15.102:0/2004340720 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe7080018f0 con 0x7fe718076610 2024-09-11T07:55:32.340 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.741+0000 7fe715ffb640 1 -- 172.21.15.102:0/2004340720 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fe708001c40 con 0x7fe718076610 2024-09-11T07:55:32.340 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.741+0000 7fe71d337640 1 -- 172.21.15.102:0/2004340720 >> v1:172.21.15.102:6789/0 conn(0x7fe718076610 legacy=0x7fe718074b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:55:32.340 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.742+0000 7fe71d337640 1 -- 172.21.15.102:0/2004340720 shutdown_connections 2024-09-11T07:55:32.340 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.742+0000 7fe71d337640 1 -- 172.21.15.102:0/2004340720 >> 172.21.15.102:0/2004340720 conn(0x7fe7180fbb90 msgr2=0x7fe7180fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:55:32.340 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.742+0000 7fe71d337640 1 -- 172.21.15.102:0/2004340720 shutdown_connections 2024-09-11T07:55:32.340 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.742+0000 7fe71d337640 1 -- 172.21.15.102:0/2004340720 wait complete. 2024-09-11T07:55:32.340 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.742+0000 7fe71d337640 1 Processor -- start 2024-09-11T07:55:32.341 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.742+0000 7fe71d337640 1 -- start start 2024-09-11T07:55:32.341 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.742+0000 7fe71d337640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe718110dd0 con 0x7fe718076610 2024-09-11T07:55:32.341 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.743+0000 7fe716ffd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fe718076610 0x7fe7181106c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:48860/0 (socket says 172.21.15.102:48860) 2024-09-11T07:55:32.341 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.743+0000 7fe716ffd640 1 -- 172.21.15.102:0/1350373110 learned_addr learned my addr 172.21.15.102:0/1350373110 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:55:32.341 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.743+0000 7fe6ff7fe640 1 -- 172.21.15.102:0/1350373110 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 997889516 0 0) 0x7fe718110dd0 con 0x7fe718076610 2024-09-11T07:55:32.341 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.743+0000 7fe6ff7fe640 1 -- 172.21.15.102:0/1350373110 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe700003d20 con 0x7fe718076610 2024-09-11T07:55:32.341 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.743+0000 7fe6ff7fe640 1 -- 172.21.15.102:0/1350373110 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3589595579 0 0) 0x7fe700003d20 con 0x7fe718076610 2024-09-11T07:55:32.341 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.743+0000 7fe6ff7fe640 1 -- 172.21.15.102:0/1350373110 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe718110dd0 con 0x7fe718076610 2024-09-11T07:55:32.341 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.743+0000 7fe6ff7fe640 1 -- 172.21.15.102:0/1350373110 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fe708001b80 con 0x7fe718076610 2024-09-11T07:55:32.341 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.743+0000 7fe6ff7fe640 1 -- 172.21.15.102:0/1350373110 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1645159823 0 0) 0x7fe718110dd0 con 0x7fe718076610 2024-09-11T07:55:32.341 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.743+0000 7fe6ff7fe640 1 -- 172.21.15.102:0/1350373110 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe718110f60 con 0x7fe718076610 2024-09-11T07:55:32.341 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.743+0000 7fe71d337640 1 -- 172.21.15.102:0/1350373110 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe718111230 con 0x7fe718076610 2024-09-11T07:55:32.341 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.743+0000 7fe71d337640 1 -- 172.21.15.102:0/1350373110 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe718114d10 con 0x7fe718076610 2024-09-11T07:55:32.342 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.743+0000 7fe6ff7fe640 1 -- 172.21.15.102:0/1350373110 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fe708004ce0 con 0x7fe718076610 2024-09-11T07:55:32.342 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.743+0000 7fe6ff7fe640 1 -- 172.21.15.102:0/1350373110 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fe708004ed0 con 0x7fe718076610 2024-09-11T07:55:32.342 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.743+0000 7fe6ff7fe640 1 -- 172.21.15.102:0/1350373110 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7fe708005330 con 0x7fe718076610 2024-09-11T07:55:32.342 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.743+0000 7fe6ff7fe640 1 -- 172.21.15.102:0/1350373110 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 1638648040 0 0) 0x7fe7080063a0 con 0x7fe718076610 2024-09-11T07:55:32.342 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.744+0000 7fe71d337640 1 -- 172.21.15.102:0/1350373110 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe704005040 con 0x7fe718076610 2024-09-11T07:55:32.342 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.746+0000 7fe6ff7fe640 1 -- 172.21.15.102:0/1350373110 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (unknown 1092875540 0 1542132831) 0x7fe708005870 con 0x7fe718076610 2024-09-11T07:55:32.342 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.803+0000 7fe71d337640 1 -- 172.21.15.102:0/1350373110 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fe704005b00 con 0x7fe718076610 2024-09-11T07:55:32.342 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.803+0000 7fe6ff7fe640 1 -- 172.21.15.102:0/1350373110 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (unknown 1155462804 0 2273960497) 0x7fe708005a90 con 0x7fe718076610 2024-09-11T07:55:32.342 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.804+0000 7fe71d337640 1 -- 172.21.15.102:0/1350373110 >> v1:172.21.15.102:6789/0 conn(0x7fe718076610 legacy=0x7fe7181106c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:55:32.342 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.804+0000 7fe71d337640 1 -- 172.21.15.102:0/1350373110 shutdown_connections 2024-09-11T07:55:32.342 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.804+0000 7fe71d337640 1 -- 172.21.15.102:0/1350373110 >> 172.21.15.102:0/1350373110 conn(0x7fe7180fbb90 msgr2=0x7fe7181054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:55:32.342 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.804+0000 7fe71d337640 1 -- 172.21.15.102:0/1350373110 shutdown_connections 2024-09-11T07:55:32.342 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:31.804+0000 7fe71d337640 1 -- 172.21.15.102:0/1350373110 wait complete. 2024-09-11T07:55:32.342 INFO:teuthology.orchestra.run.smithi102.stdout:mon is available 2024-09-11T07:55:32.343 INFO:teuthology.orchestra.run.smithi102.stdout:Assimilating anything we can from ceph.conf... 2024-09-11T07:55:32.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:32 smithi102 ceph-mon[29379]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T07:55:32.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:32 smithi102 ceph-mon[29379]: monmap e1: 1 mons at {a=v1:172.21.15.102:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:55:32.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:32 smithi102 ceph-mon[29379]: fsmap 2024-09-11T07:55:32.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:32 smithi102 ceph-mon[29379]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T07:55:32.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:32 smithi102 ceph-mon[29379]: mgrmap e1: no daemons active 2024-09-11T07:55:32.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:32 smithi102 ceph-mon[29379]: from='client.? v1:172.21.15.102:0/1350373110' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-11T07:55:33.351 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:33 smithi102 ceph-mon[29379]: from='client.? v1:172.21.15.102:0/940769072' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T07:55:33.351 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:33 smithi102 ceph-mon[29379]: from='client.? v1:172.21.15.102:0/940769072' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-11T07:55:33.358 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-09-11T07:55:33.359 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T07:55:33.359 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout fsid = 2051892c-7013-11ef-bcea-c7b262605968 2024-09-11T07:55:33.359 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T07:55:33.359 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_host = [v1:172.21.15.102:6789] 2024-09-11T07:55:33.359 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T07:55:33.359 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T07:55:33.359 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T07:55:33.359 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T07:55:33.359 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-09-11T07:55:33.359 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T07:55:33.359 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T07:55:33.359 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-09-11T07:55:33.360 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T07:55:33.360 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T07:55:33.360 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T07:55:33.360 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.754+0000 7f347ea41640 1 Processor -- start 2024-09-11T07:55:33.360 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.754+0000 7f347ea41640 1 -- start start 2024-09-11T07:55:33.360 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.754+0000 7f347ea41640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f34780ffd70 con 0x7f34780ff250 2024-09-11T07:55:33.360 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.754+0000 7f3477fff640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f34780ff250 0x7f34780ff660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:48876/0 (socket says 172.21.15.102:48876) 2024-09-11T07:55:33.360 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.754+0000 7f3477fff640 1 -- 172.21.15.102:0/2662582267 learned_addr learned my addr 172.21.15.102:0/2662582267 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:55:33.360 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.755+0000 7f3476ffd640 1 -- 172.21.15.102:0/2662582267 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3846387569 0 0) 0x7f34780ffd70 con 0x7f34780ff250 2024-09-11T07:55:33.360 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.755+0000 7f3476ffd640 1 -- 172.21.15.102:0/2662582267 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34600034e0 con 0x7f34780ff250 2024-09-11T07:55:33.360 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.755+0000 7f3476ffd640 1 -- 172.21.15.102:0/2662582267 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4049981072 0 0) 0x7f34600034e0 con 0x7f34780ff250 2024-09-11T07:55:33.360 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.755+0000 7f3476ffd640 1 -- 172.21.15.102:0/2662582267 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3478104710 con 0x7f34780ff250 2024-09-11T07:55:33.361 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.755+0000 7f3476ffd640 1 -- 172.21.15.102:0/2662582267 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f3468001590 con 0x7f34780ff250 2024-09-11T07:55:33.361 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.755+0000 7f3476ffd640 1 -- 172.21.15.102:0/2662582267 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f34680018f0 con 0x7f34780ff250 2024-09-11T07:55:33.361 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.755+0000 7f3476ffd640 1 -- 172.21.15.102:0/2662582267 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f3468001c40 con 0x7f34780ff250 2024-09-11T07:55:33.361 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.755+0000 7f347ea41640 1 -- 172.21.15.102:0/2662582267 >> v1:172.21.15.102:6789/0 conn(0x7f34780ff250 legacy=0x7f34780ff660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:55:33.361 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.755+0000 7f347ea41640 1 -- 172.21.15.102:0/2662582267 shutdown_connections 2024-09-11T07:55:33.361 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.755+0000 7f347ea41640 1 -- 172.21.15.102:0/2662582267 >> 172.21.15.102:0/2662582267 conn(0x7f34780fa9d0 msgr2=0x7f34780fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:55:33.361 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.755+0000 7f347ea41640 1 -- 172.21.15.102:0/2662582267 shutdown_connections 2024-09-11T07:55:33.361 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.755+0000 7f347ea41640 1 -- 172.21.15.102:0/2662582267 wait complete. 2024-09-11T07:55:33.361 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.756+0000 7f347ea41640 1 Processor -- start 2024-09-11T07:55:33.361 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.756+0000 7f347ea41640 1 -- start start 2024-09-11T07:55:33.361 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.756+0000 7f347ea41640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f347810eba0 con 0x7f34780ff250 2024-09-11T07:55:33.361 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.756+0000 7f3477fff640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f34780ff250 0x7f347810e490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:48886/0 (socket says 172.21.15.102:48886) 2024-09-11T07:55:33.361 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.756+0000 7f3477fff640 1 -- 172.21.15.102:0/940769072 learned_addr learned my addr 172.21.15.102:0/940769072 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:55:33.362 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.757+0000 7f3474ff9640 1 -- 172.21.15.102:0/940769072 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 676252070 0 0) 0x7f347810eba0 con 0x7f34780ff250 2024-09-11T07:55:33.362 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.757+0000 7f3474ff9640 1 -- 172.21.15.102:0/940769072 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3460003d20 con 0x7f34780ff250 2024-09-11T07:55:33.362 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.757+0000 7f3474ff9640 1 -- 172.21.15.102:0/940769072 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 370689425 0 0) 0x7f3460003d20 con 0x7f34780ff250 2024-09-11T07:55:33.362 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.757+0000 7f3474ff9640 1 -- 172.21.15.102:0/940769072 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f347810eba0 con 0x7f34780ff250 2024-09-11T07:55:33.362 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.757+0000 7f3474ff9640 1 -- 172.21.15.102:0/940769072 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f3468001550 con 0x7f34780ff250 2024-09-11T07:55:33.362 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.757+0000 7f3474ff9640 1 -- 172.21.15.102:0/940769072 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 735796168 0 0) 0x7f347810eba0 con 0x7f34780ff250 2024-09-11T07:55:33.362 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.757+0000 7f3474ff9640 1 -- 172.21.15.102:0/940769072 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347810ed30 con 0x7f34780ff250 2024-09-11T07:55:33.362 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.757+0000 7f347ea41640 1 -- 172.21.15.102:0/940769072 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f347810f000 con 0x7f34780ff250 2024-09-11T07:55:33.362 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.757+0000 7f347ea41640 1 -- 172.21.15.102:0/940769072 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3478112ae0 con 0x7f34780ff250 2024-09-11T07:55:33.362 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.757+0000 7f3474ff9640 1 -- 172.21.15.102:0/940769072 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f3468004c70 con 0x7f34780ff250 2024-09-11T07:55:33.362 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.757+0000 7f3474ff9640 1 -- 172.21.15.102:0/940769072 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f3468004f80 con 0x7f34780ff250 2024-09-11T07:55:33.362 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.757+0000 7f3474ff9640 1 -- 172.21.15.102:0/940769072 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7f34680053a0 con 0x7f34780ff250 2024-09-11T07:55:33.362 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.757+0000 7f3474ff9640 1 -- 172.21.15.102:0/940769072 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 1638648040 0 0) 0x7f34680063b0 con 0x7f34780ff250 2024-09-11T07:55:33.363 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.758+0000 7f347ea41640 1 -- 172.21.15.102:0/940769072 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3464005040 con 0x7f34780ff250 2024-09-11T07:55:33.363 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.760+0000 7f3474ff9640 1 -- 172.21.15.102:0/940769072 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (unknown 1092875540 0 1542132831) 0x7f3468005830 con 0x7f34780ff250 2024-09-11T07:55:33.363 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.815+0000 7f347ea41640 1 -- 172.21.15.102:0/940769072 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f3464005870 con 0x7f34780ff250 2024-09-11T07:55:33.363 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.818+0000 7f3474ff9640 1 -- 172.21.15.102:0/940769072 <== mon.0 v1:172.21.15.102:6789/0 10 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7f3468005a50 con 0x7f34780ff250 2024-09-11T07:55:33.363 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.818+0000 7f3474ff9640 1 -- 172.21.15.102:0/940769072 <== mon.0 v1:172.21.15.102:6789/0 11 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+354 (unknown 1213389831 0 1145609554) 0x7f34680073b0 con 0x7f34780ff250 2024-09-11T07:55:33.363 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.819+0000 7f347ea41640 1 -- 172.21.15.102:0/940769072 >> v1:172.21.15.102:6789/0 conn(0x7f34780ff250 legacy=0x7f347810e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:55:33.363 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.819+0000 7f347ea41640 1 -- 172.21.15.102:0/940769072 shutdown_connections 2024-09-11T07:55:33.363 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.819+0000 7f347ea41640 1 -- 172.21.15.102:0/940769072 >> 172.21.15.102:0/940769072 conn(0x7f34780fa9d0 msgr2=0x7f3478073870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:55:33.363 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.819+0000 7f347ea41640 1 -- 172.21.15.102:0/940769072 shutdown_connections 2024-09-11T07:55:33.363 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:32.819+0000 7f347ea41640 1 -- 172.21.15.102:0/940769072 wait complete. 2024-09-11T07:55:33.363 INFO:teuthology.orchestra.run.smithi102.stdout:Generating new minimal ceph.conf... 2024-09-11T07:55:34.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:34 smithi102 ceph-mon[29379]: from='client.? v1:172.21.15.102:0/1047449457' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:55:34.431 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.760+0000 7f06a512f640 1 Processor -- start 2024-09-11T07:55:34.431 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.760+0000 7f06a512f640 1 -- start start 2024-09-11T07:55:34.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.760+0000 7f06a512f640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f06a00ffd70 con 0x7f06a00ff250 2024-09-11T07:55:34.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.760+0000 7f069ed9d640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f06a00ff250 0x7f06a00ff660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:48890/0 (socket says 172.21.15.102:48890) 2024-09-11T07:55:34.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.761+0000 7f069ed9d640 1 -- 172.21.15.102:0/4160731446 learned_addr learned my addr 172.21.15.102:0/4160731446 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:55:34.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.761+0000 7f069dd9b640 1 -- 172.21.15.102:0/4160731446 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2429559 0 0) 0x7f06a00ffd70 con 0x7f06a00ff250 2024-09-11T07:55:34.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.761+0000 7f069dd9b640 1 -- 172.21.15.102:0/4160731446 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06880034e0 con 0x7f06a00ff250 2024-09-11T07:55:34.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.761+0000 7f069dd9b640 1 -- 172.21.15.102:0/4160731446 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 176403950 0 0) 0x7f06880034e0 con 0x7f06a00ff250 2024-09-11T07:55:34.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.761+0000 7f069dd9b640 1 -- 172.21.15.102:0/4160731446 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a0104710 con 0x7f06a00ff250 2024-09-11T07:55:34.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.761+0000 7f069dd9b640 1 -- 172.21.15.102:0/4160731446 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f0690001590 con 0x7f06a00ff250 2024-09-11T07:55:34.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.761+0000 7f069dd9b640 1 -- 172.21.15.102:0/4160731446 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7f0690001cb0 con 0x7f06a00ff250 2024-09-11T07:55:34.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.761+0000 7f069dd9b640 1 -- 172.21.15.102:0/4160731446 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f0690005670 con 0x7f06a00ff250 2024-09-11T07:55:34.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.761+0000 7f06a512f640 1 -- 172.21.15.102:0/4160731446 >> v1:172.21.15.102:6789/0 conn(0x7f06a00ff250 legacy=0x7f06a00ff660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:55:34.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.762+0000 7f06a512f640 1 -- 172.21.15.102:0/4160731446 shutdown_connections 2024-09-11T07:55:34.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.762+0000 7f06a512f640 1 -- 172.21.15.102:0/4160731446 >> 172.21.15.102:0/4160731446 conn(0x7f06a00fa9d0 msgr2=0x7f06a00fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:55:34.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.762+0000 7f06a512f640 1 -- 172.21.15.102:0/4160731446 shutdown_connections 2024-09-11T07:55:34.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.762+0000 7f06a512f640 1 -- 172.21.15.102:0/4160731446 wait complete. 2024-09-11T07:55:34.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.762+0000 7f06a512f640 1 Processor -- start 2024-09-11T07:55:34.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.762+0000 7f06a512f640 1 -- start start 2024-09-11T07:55:34.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.762+0000 7f06a512f640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f06a00fceb0 con 0x7f06a00ff250 2024-09-11T07:55:34.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.762+0000 7f069ed9d640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f06a00ff250 0x7f06a00fc7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:48906/0 (socket says 172.21.15.102:48906) 2024-09-11T07:55:34.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.762+0000 7f069ed9d640 1 -- 172.21.15.102:0/1047449457 learned_addr learned my addr 172.21.15.102:0/1047449457 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:55:34.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.763+0000 7f06877fe640 1 -- 172.21.15.102:0/1047449457 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 421468078 0 0) 0x7f06a00fceb0 con 0x7f06a00ff250 2024-09-11T07:55:34.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.763+0000 7f06877fe640 1 -- 172.21.15.102:0/1047449457 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06880053f0 con 0x7f06a00ff250 2024-09-11T07:55:34.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.763+0000 7f06877fe640 1 -- 172.21.15.102:0/1047449457 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 720566458 0 0) 0x7f06880053f0 con 0x7f06a00ff250 2024-09-11T07:55:34.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.763+0000 7f06877fe640 1 -- 172.21.15.102:0/1047449457 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f06a00fceb0 con 0x7f06a00ff250 2024-09-11T07:55:34.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.763+0000 7f06877fe640 1 -- 172.21.15.102:0/1047449457 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f06900047a0 con 0x7f06a00ff250 2024-09-11T07:55:34.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.763+0000 7f06877fe640 1 -- 172.21.15.102:0/1047449457 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 162622836 0 0) 0x7f06a00fceb0 con 0x7f06a00ff250 2024-09-11T07:55:34.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.763+0000 7f06877fe640 1 -- 172.21.15.102:0/1047449457 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a0115d20 con 0x7f06a00ff250 2024-09-11T07:55:34.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.763+0000 7f06a512f640 1 -- 172.21.15.102:0/1047449457 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a0114d70 con 0x7f06a00ff250 2024-09-11T07:55:34.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.763+0000 7f06a512f640 1 -- 172.21.15.102:0/1047449457 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a0115240 con 0x7f06a00ff250 2024-09-11T07:55:34.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.763+0000 7f06877fe640 1 -- 172.21.15.102:0/1047449457 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7f06900049e0 con 0x7f06a00ff250 2024-09-11T07:55:34.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.763+0000 7f06877fe640 1 -- 172.21.15.102:0/1047449457 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f0690006060 con 0x7f06a00ff250 2024-09-11T07:55:34.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.763+0000 7f06877fe640 1 -- 172.21.15.102:0/1047449457 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7f0690006500 con 0x7f06a00ff250 2024-09-11T07:55:34.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.763+0000 7f06877fe640 1 -- 172.21.15.102:0/1047449457 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 1638648040 0 0) 0x7f0690007520 con 0x7f06a00ff250 2024-09-11T07:55:34.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.764+0000 7f06a512f640 1 -- 172.21.15.102:0/1047449457 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f068c005040 con 0x7f06a00ff250 2024-09-11T07:55:34.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.766+0000 7f06877fe640 1 -- 172.21.15.102:0/1047449457 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (unknown 1092875540 0 1542132831) 0x7f06900059b0 con 0x7f06a00ff250 2024-09-11T07:55:34.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.821+0000 7f06a512f640 1 -- 172.21.15.102:0/1047449457 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f068c0052f0 con 0x7f06a00ff250 2024-09-11T07:55:34.435 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.822+0000 7f06877fe640 1 -- 172.21.15.102:0/1047449457 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+148 (unknown 1452402520 0 253242038) 0x7f0690004ba0 con 0x7f06a00ff250 2024-09-11T07:55:34.435 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.822+0000 7f06a512f640 1 -- 172.21.15.102:0/1047449457 >> v1:172.21.15.102:6789/0 conn(0x7f06a00ff250 legacy=0x7f06a00fc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:55:34.435 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.822+0000 7f06a512f640 1 -- 172.21.15.102:0/1047449457 shutdown_connections 2024-09-11T07:55:34.435 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.822+0000 7f06a512f640 1 -- 172.21.15.102:0/1047449457 >> 172.21.15.102:0/1047449457 conn(0x7f06a00fa9d0 msgr2=0x7f06a0072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:55:34.435 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.822+0000 7f06a512f640 1 -- 172.21.15.102:0/1047449457 shutdown_connections 2024-09-11T07:55:34.435 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:33.822+0000 7f06a512f640 1 -- 172.21.15.102:0/1047449457 wait complete. 2024-09-11T07:55:34.435 INFO:teuthology.orchestra.run.smithi102.stdout:Restarting the monitor... 2024-09-11T07:55:34.705 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:34 smithi102 systemd[1]: Stopping Ceph mon.a for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T07:55:35.060 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:34 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-a[29356]: 2024-09-11T07:55:34.702+0000 7fbd3f1c3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:55:35.060 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:34 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-a[29356]: 2024-09-11T07:55:34.702+0000 7fbd3f1c3640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-11T07:55:35.060 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:34 smithi102 podman[29696]: 2024-09-11 07:55:34.78524561 +0000 UTC m=+0.198917425 container died 3c2d329aca3d833b0d8a3749f884c3ab1c2cbc8194c34401a52faedf302f5cf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:55:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 podman[29696]: 2024-09-11 07:55:35.159167879 +0000 UTC m=+0.572839695 container cleanup 3c2d329aca3d833b0d8a3749f884c3ab1c2cbc8194c34401a52faedf302f5cf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-11T07:55:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 bash[29696]: ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-a 2024-09-11T07:55:35.728 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 podman[29713]: 2024-09-11 07:55:35.401703054 +0000 UTC m=+0.614839476 container remove 3c2d329aca3d833b0d8a3749f884c3ab1c2cbc8194c34401a52faedf302f5cf9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T07:55:35.728 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T07:55:35.728 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 systemd[1]: Stopped Ceph mon.a for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T07:55:35.728 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 systemd[1]: Starting Ceph mon.a for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T07:55:35.979 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 podman[29811]: 2024-09-11 07:55:35.72775979 +0000 UTC m=+0.108183989 container create 36c0f168f3ce4234c5bf7d804f02edd9522617f37391a434b8e119cf6dc4fa24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-a, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:55:35.979 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 podman[29811]: 2024-09-11 07:55:35.635171913 +0000 UTC m=+0.015596063 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:55:35.979 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 podman[29811]: 2024-09-11 07:55:35.955077137 +0000 UTC m=+0.335501287 container init 36c0f168f3ce4234c5bf7d804f02edd9522617f37391a434b8e119cf6dc4fa24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-11T07:55:35.979 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 podman[29811]: 2024-09-11 07:55:35.959825282 +0000 UTC m=+0.340249432 container start 36c0f168f3ce4234c5bf7d804f02edd9522617f37391a434b8e119cf6dc4fa24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T07:55:35.979 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T07:55:35.979 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable), process ceph-mon, pid 7 2024-09-11T07:55:35.979 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: pidfile_write: ignore empty --pid-file 2024-09-11T07:55:36.060 INFO:teuthology.orchestra.run.smithi102.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-11T07:55:36.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: load: jerasure load: lrc 2024-09-11T07:55:36.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: RocksDB version: 6.15.5 2024-09-11T07:55:36.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-11T07:55:36.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Compile date Sep 10 2024 2024-09-11T07:55:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: DB SUMMARY 2024-09-11T07:55:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: DB Session ID: 1BK6X0UP3NZKVFJ9B74W 2024-09-11T07:55:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: CURRENT file: CURRENT 2024-09-11T07:55:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T07:55:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-09-11T07:55:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-11T07:55:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 83794 ; 2024-09-11T07:55:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.error_if_exists: 0 2024-09-11T07:55:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.create_if_missing: 0 2024-09-11T07:55:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.paranoid_checks: 1 2024-09-11T07:55:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T07:55:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.env: 0x55e8ad533440 2024-09-11T07:55:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.fs: Posix File System 2024-09-11T07:55:36.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.info_log: 0x55e8adf72e60 2024-09-11T07:55:36.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T07:55:36.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.statistics: (nil) 2024-09-11T07:55:36.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.use_fsync: 0 2024-09-11T07:55:36.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_log_file_size: 0 2024-09-11T07:55:36.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T07:55:36.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T07:55:36.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T07:55:36.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T07:55:36.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.allow_fallocate: 1 2024-09-11T07:55:36.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T07:55:36.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T07:55:36.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.use_direct_reads: 0 2024-09-11T07:55:36.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T07:55:36.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T07:55:36.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.db_log_dir: 2024-09-11T07:55:36.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-09-11T07:55:36.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T07:55:36.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T07:55:36.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T07:55:36.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T07:55:36.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T07:55:36.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T07:55:36.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T07:55:36.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T07:55:36.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.write_buffer_manager: 0x55e8ae030780 2024-09-11T07:55:36.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T07:55:36.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-11T07:55:36.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T07:55:36.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T07:55:36.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T07:55:36.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T07:55:36.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T07:55:36.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T07:55:36.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T07:55:36.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.unordered_write: 0 2024-09-11T07:55:36.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T07:55:36.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T07:55:36.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T07:55:36.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T07:55:36.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.row_cache: None 2024-09-11T07:55:36.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.wal_filter: None 2024-09-11T07:55:36.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T07:55:36.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T07:55:36.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.preserve_deletes: 0 2024-09-11T07:55:36.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.two_write_queues: 0 2024-09-11T07:55:36.403 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T07:55:36.403 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.atomic_flush: 0 2024-09-11T07:55:36.403 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T07:55:36.403 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T07:55:36.403 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T07:55:36.403 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.log_readahead_size: 0 2024-09-11T07:55:36.403 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T07:55:36.403 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T07:55:36.403 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T07:55:36.403 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T07:55:36.403 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T07:55:36.403 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T07:55:36.403 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_background_jobs: 2 2024-09-11T07:55:36.404 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_background_compactions: -1 2024-09-11T07:55:36.404 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_subcompactions: 1 2024-09-11T07:55:36.404 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T07:55:36.404 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T07:55:36.404 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T07:55:36.404 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T07:55:36.404 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T07:55:36.404 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T07:55:36.404 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T07:55:36.404 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T07:55:36.404 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_open_files: -1 2024-09-11T07:55:36.404 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T07:55:36.405 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T07:55:36.405 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T07:55:36.405 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T07:55:36.405 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_background_flushes: -1 2024-09-11T07:55:36.405 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Compression algorithms supported: 2024-09-11T07:55:36.405 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: kZSTD supported: 0 2024-09-11T07:55:36.405 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: kXpressCompression supported: 0 2024-09-11T07:55:36.405 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: kBZip2Compression supported: 0 2024-09-11T07:55:36.405 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T07:55:36.405 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: kLZ4Compression supported: 1 2024-09-11T07:55:36.405 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: kZlibCompression supported: 1 2024-09-11T07:55:36.405 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T07:55:36.406 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: kSnappyCompression supported: 1 2024-09-11T07:55:36.406 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T07:55:36.406 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2024-09-11T07:55:36.406 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-11T07:55:36.406 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T07:55:36.406 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.merge_operator: 2024-09-11T07:55:36.406 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.compaction_filter: None 2024-09-11T07:55:36.406 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T07:55:36.406 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T07:55:36.406 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T07:55:36.406 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T07:55:36.406 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e8adf19a38) 2024-09-11T07:55:36.407 INFO:journalctl@ceph.mon.a.smithi102.stdout: cache_index_and_filter_blocks: 1 2024-09-11T07:55:36.407 INFO:journalctl@ceph.mon.a.smithi102.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T07:55:36.407 INFO:journalctl@ceph.mon.a.smithi102.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T07:55:36.407 INFO:journalctl@ceph.mon.a.smithi102.stdout: pin_top_level_index_and_filter: 1 2024-09-11T07:55:36.407 INFO:journalctl@ceph.mon.a.smithi102.stdout: index_type: 0 2024-09-11T07:55:36.407 INFO:journalctl@ceph.mon.a.smithi102.stdout: data_block_index_type: 0 2024-09-11T07:55:36.407 INFO:journalctl@ceph.mon.a.smithi102.stdout: index_shortening: 1 2024-09-11T07:55:36.407 INFO:journalctl@ceph.mon.a.smithi102.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T07:55:36.407 INFO:journalctl@ceph.mon.a.smithi102.stdout: hash_index_allow_collision: 1 2024-09-11T07:55:36.407 INFO:journalctl@ceph.mon.a.smithi102.stdout: checksum: 1 2024-09-11T07:55:36.407 INFO:journalctl@ceph.mon.a.smithi102.stdout: no_block_cache: 0 2024-09-11T07:55:36.407 INFO:journalctl@ceph.mon.a.smithi102.stdout: block_cache: 0x55e8adf4f1f0 2024-09-11T07:55:36.407 INFO:journalctl@ceph.mon.a.smithi102.stdout: block_cache_name: BinnedLRUCache 2024-09-11T07:55:36.408 INFO:journalctl@ceph.mon.a.smithi102.stdout: block_cache_options: 2024-09-11T07:55:36.408 INFO:journalctl@ceph.mon.a.smithi102.stdout: capacity : 536870912 2024-09-11T07:55:36.408 INFO:journalctl@ceph.mon.a.smithi102.stdout: num_shard_bits : 4 2024-09-11T07:55:36.408 INFO:journalctl@ceph.mon.a.smithi102.stdout: strict_capacity_limit : 0 2024-09-11T07:55:36.408 INFO:journalctl@ceph.mon.a.smithi102.stdout: high_pri_pool_ratio: 0.000 2024-09-11T07:55:36.408 INFO:journalctl@ceph.mon.a.smithi102.stdout: block_cache_compressed: (nil) 2024-09-11T07:55:36.408 INFO:journalctl@ceph.mon.a.smithi102.stdout: persistent_cache: (nil) 2024-09-11T07:55:36.408 INFO:journalctl@ceph.mon.a.smithi102.stdout: block_size: 4096 2024-09-11T07:55:36.408 INFO:journalctl@ceph.mon.a.smithi102.stdout: block_size_deviation: 10 2024-09-11T07:55:36.408 INFO:journalctl@ceph.mon.a.smithi102.stdout: block_restart_interval: 16 2024-09-11T07:55:36.408 INFO:journalctl@ceph.mon.a.smithi102.stdout: index_block_restart_interval: 1 2024-09-11T07:55:36.408 INFO:journalctl@ceph.mon.a.smithi102.stdout: metadata_block_size: 4096 2024-09-11T07:55:36.408 INFO:journalctl@ceph.mon.a.smithi102.stdout: partition_filters: 0 2024-09-11T07:55:36.408 INFO:journalctl@ceph.mon.a.smithi102.stdout: use_delta_encoding: 1 2024-09-11T07:55:36.409 INFO:journalctl@ceph.mon.a.smithi102.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-11T07:55:36.409 INFO:journalctl@ceph.mon.a.smithi102.stdout: whole_key_filtering: 1 2024-09-11T07:55:36.409 INFO:journalctl@ceph.mon.a.smithi102.stdout: verify_compression: 0 2024-09-11T07:55:36.409 INFO:journalctl@ceph.mon.a.smithi102.stdout: read_amp_bytes_per_bit: 0 2024-09-11T07:55:36.409 INFO:journalctl@ceph.mon.a.smithi102.stdout: format_version: 4 2024-09-11T07:55:36.409 INFO:journalctl@ceph.mon.a.smithi102.stdout: enable_index_compression: 1 2024-09-11T07:55:36.409 INFO:journalctl@ceph.mon.a.smithi102.stdout: block_align: 0 2024-09-11T07:55:36.409 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T07:55:36.409 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T07:55:36.409 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.compression: NoCompression 2024-09-11T07:55:36.409 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T07:55:36.409 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T07:55:36.410 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T07:55:36.410 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.num_levels: 7 2024-09-11T07:55:36.410 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T07:55:36.410 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T07:55:36.410 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T07:55:36.410 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T07:55:36.410 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T07:55:36.410 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T07:55:36.410 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T07:55:36.410 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T07:55:36.410 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T07:55:36.410 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T07:55:36.410 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T07:55:36.411 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T07:55:36.411 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T07:55:36.411 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T07:55:36.411 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T07:55:36.411 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T07:55:36.411 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T07:55:36.411 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T07:55:36.411 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T07:55:36.411 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T07:55:36.411 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T07:55:36.411 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T07:55:36.411 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T07:55:36.411 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T07:55:36.412 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T07:55:36.412 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T07:55:36.412 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T07:55:36.412 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T07:55:36.412 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T07:55:36.412 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T07:55:36.412 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T07:55:36.412 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T07:55:36.412 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T07:55:36.412 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T07:55:36.412 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.arena_block_size: 4194304 2024-09-11T07:55:36.412 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T07:55:36.412 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T07:55:36.413 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-11T07:55:36.413 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T07:55:36.413 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T07:55:36.413 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T07:55:36.413 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T07:55:36.413 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T07:55:36.413 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T07:55:36.413 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T07:55:36.413 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T07:55:36.413 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T07:55:36.413 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T07:55:36.413 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T07:55:36.413 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T07:55:36.414 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.inplace_update_support: 0 2024-09-11T07:55:36.414 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T07:55:36.414 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T07:55:36.414 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T07:55:36.414 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T07:55:36.414 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.bloom_locality: 0 2024-09-11T07:55:36.414 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.max_successive_merges: 0 2024-09-11T07:55:36.414 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T07:55:36.414 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T07:55:36.414 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T07:55:36.414 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T07:55:36.414 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.ttl: 2592000 2024-09-11T07:55:36.414 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T07:55:36.415 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.enable_blob_files: false 2024-09-11T07:55:36.415 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.min_blob_size: 0 2024-09-11T07:55:36.415 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T07:55:36.415 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T07:55:36.415 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T07:55:36.415 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T07:55:36.415 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-11T07:55:36.415 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-09-11T07:55:36.415 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-09-11T07:55:36.415 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041335983249, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-09-11T07:55:36.415 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-09-11T07:55:36.415 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-11T07:55:36.415 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041335983979, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 80176, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 78442, "index_size": 196, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10435, "raw_average_key_size": 47, "raw_value_size": 72551, "raw_average_value_size": 331, "num_data_blocks": 10, "num_entries": 219, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1726041335, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "4318ab3d-ef84-448b-a824-d8e897729a8b", "db_session_id": "1BK6X0UP3NZKVFJ9B74W"}} 2024-09-11T07:55:36.416 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-09-11T07:55:36.416 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041335984311, "job": 1, "event": "recovery_finished"} 2024-09-11T07:55:36.416 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T07:55:36.416 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55e8adf82700 2024-09-11T07:55:36.416 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: DB pointer 0x55e8adf52000 2024-09-11T07:55:36.416 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-11T07:55:36.416 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-11T07:55:36.416 INFO:journalctl@ceph.mon.a.smithi102.stdout: ** DB Stats ** 2024-09-11T07:55:36.416 INFO:journalctl@ceph.mon.a.smithi102.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:55:36.416 INFO:journalctl@ceph.mon.a.smithi102.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T07:55:36.416 INFO:journalctl@ceph.mon.a.smithi102.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T07:55:36.416 INFO:journalctl@ceph.mon.a.smithi102.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T07:55:36.416 INFO:journalctl@ceph.mon.a.smithi102.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T07:55:36.416 INFO:journalctl@ceph.mon.a.smithi102.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-11T07:55:36.417 INFO:journalctl@ceph.mon.a.smithi102.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T07:55:36.417 INFO:journalctl@ceph.mon.a.smithi102.stdout: 2024-09-11T07:55:36.417 INFO:journalctl@ceph.mon.a.smithi102.stdout: ** Compaction Stats [default] ** 2024-09-11T07:55:36.417 INFO:journalctl@ceph.mon.a.smithi102.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:55:36.417 INFO:journalctl@ceph.mon.a.smithi102.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:55:36.417 INFO:journalctl@ceph.mon.a.smithi102.stdout: L0 2/0 80.05 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 154.5 0.00 0.00 1 0.000 0 0 2024-09-11T07:55:36.417 INFO:journalctl@ceph.mon.a.smithi102.stdout: Sum 2/0 80.05 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 154.5 0.00 0.00 1 0.000 0 0 2024-09-11T07:55:36.417 INFO:journalctl@ceph.mon.a.smithi102.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 154.5 0.00 0.00 1 0.000 0 0 2024-09-11T07:55:36.417 INFO:journalctl@ceph.mon.a.smithi102.stdout: 2024-09-11T07:55:36.417 INFO:journalctl@ceph.mon.a.smithi102.stdout: ** Compaction Stats [default] ** 2024-09-11T07:55:36.417 INFO:journalctl@ceph.mon.a.smithi102.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:55:36.417 INFO:journalctl@ceph.mon.a.smithi102.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:55:36.417 INFO:journalctl@ceph.mon.a.smithi102.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 154.5 0.00 0.00 1 0.000 0 0 2024-09-11T07:55:36.418 INFO:journalctl@ceph.mon.a.smithi102.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:55:36.418 INFO:journalctl@ceph.mon.a.smithi102.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T07:55:36.418 INFO:journalctl@ceph.mon.a.smithi102.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T07:55:36.418 INFO:journalctl@ceph.mon.a.smithi102.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T07:55:36.418 INFO:journalctl@ceph.mon.a.smithi102.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T07:55:36.418 INFO:journalctl@ceph.mon.a.smithi102.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T07:55:36.418 INFO:journalctl@ceph.mon.a.smithi102.stdout: Cumulative compaction: 0.00 GB write, 25.39 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:55:36.418 INFO:journalctl@ceph.mon.a.smithi102.stdout: Interval compaction: 0.00 GB write, 25.39 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:55:36.418 INFO:journalctl@ceph.mon.a.smithi102.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T07:55:36.418 INFO:journalctl@ceph.mon.a.smithi102.stdout: 2024-09-11T07:55:36.418 INFO:journalctl@ceph.mon.a.smithi102.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T07:55:36.418 INFO:journalctl@ceph.mon.a.smithi102.stdout: 2024-09-11T07:55:36.418 INFO:journalctl@ceph.mon.a.smithi102.stdout: ** Compaction Stats [default] ** 2024-09-11T07:55:36.419 INFO:journalctl@ceph.mon.a.smithi102.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:55:36.419 INFO:journalctl@ceph.mon.a.smithi102.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:55:36.419 INFO:journalctl@ceph.mon.a.smithi102.stdout: L0 2/0 80.05 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 154.5 0.00 0.00 1 0.000 0 0 2024-09-11T07:55:36.419 INFO:journalctl@ceph.mon.a.smithi102.stdout: Sum 2/0 80.05 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 154.5 0.00 0.00 1 0.000 0 0 2024-09-11T07:55:36.419 INFO:journalctl@ceph.mon.a.smithi102.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-11T07:55:36.419 INFO:journalctl@ceph.mon.a.smithi102.stdout: 2024-09-11T07:55:36.419 INFO:journalctl@ceph.mon.a.smithi102.stdout: ** Compaction Stats [default] ** 2024-09-11T07:55:36.419 INFO:journalctl@ceph.mon.a.smithi102.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:55:36.419 INFO:journalctl@ceph.mon.a.smithi102.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:55:36.419 INFO:journalctl@ceph.mon.a.smithi102.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 154.5 0.00 0.00 1 0.000 0 0 2024-09-11T07:55:36.419 INFO:journalctl@ceph.mon.a.smithi102.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:55:36.419 INFO:journalctl@ceph.mon.a.smithi102.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T07:55:36.419 INFO:journalctl@ceph.mon.a.smithi102.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T07:55:36.420 INFO:journalctl@ceph.mon.a.smithi102.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T07:55:36.420 INFO:journalctl@ceph.mon.a.smithi102.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T07:55:36.420 INFO:journalctl@ceph.mon.a.smithi102.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T07:55:36.420 INFO:journalctl@ceph.mon.a.smithi102.stdout: Cumulative compaction: 0.00 GB write, 25.14 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:55:36.420 INFO:journalctl@ceph.mon.a.smithi102.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:55:36.420 INFO:journalctl@ceph.mon.a.smithi102.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T07:55:36.420 INFO:journalctl@ceph.mon.a.smithi102.stdout: 2024-09-11T07:55:36.420 INFO:journalctl@ceph.mon.a.smithi102.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T07:55:36.420 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: starting mon.a rank 0 at public addrs v1:172.21.15.102:6789/0 at bind addrs v1:172.21.15.102:6789/0 mon_data /var/lib/ceph/mon/ceph-a fsid 2051892c-7013-11ef-bcea-c7b262605968 2024-09-11T07:55:36.420 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: mon.a@-1(???) e1 preinit fsid 2051892c-7013-11ef-bcea-c7b262605968 2024-09-11T07:55:36.420 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: mon.a@-1(???).mds e1 new map 2024-09-11T07:55:36.420 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: mon.a@-1(???).mds e1 print_map 2024-09-11T07:55:36.420 INFO:journalctl@ceph.mon.a.smithi102.stdout: e1 2024-09-11T07:55:36.421 INFO:journalctl@ceph.mon.a.smithi102.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T07:55:36.421 INFO:journalctl@ceph.mon.a.smithi102.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-11T07:55:36.421 INFO:journalctl@ceph.mon.a.smithi102.stdout: legacy client fscid: -1 2024-09-11T07:55:36.421 INFO:journalctl@ceph.mon.a.smithi102.stdout: 2024-09-11T07:55:36.421 INFO:journalctl@ceph.mon.a.smithi102.stdout: No filesystems configured 2024-09-11T07:55:36.421 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T07:55:36.421 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:55:36.421 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:55:36.421 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:55:36.421 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:35 smithi102 ceph-mon[29849]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-11T07:55:36.421 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:36 smithi102 ceph-mon[29849]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T07:55:36.421 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:36 smithi102 ceph-mon[29849]: monmap e1: 1 mons at {a=v1:172.21.15.102:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:55:36.421 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:36 smithi102 ceph-mon[29849]: fsmap 2024-09-11T07:55:36.422 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:36 smithi102 ceph-mon[29849]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T07:55:36.422 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:36 smithi102 ceph-mon[29849]: mgrmap e1: no daemons active 2024-09-11T07:55:36.422 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:36 smithi102 bash[29811]: 36c0f168f3ce4234c5bf7d804f02edd9522617f37391a434b8e119cf6dc4fa24 2024-09-11T07:55:36.422 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:36 smithi102 systemd[1]: Started Ceph mon.a for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T07:55:39.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:39 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2864057001' entity='client.admin' 2024-09-11T07:55:45.019 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.332+0000 7fafbbd08640 1 Processor -- start 2024-09-11T07:55:45.019 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.332+0000 7fafbbd08640 1 -- start start 2024-09-11T07:55:45.019 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.332+0000 7fafbbd08640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fafb4075250 con 0x7fafb4076620 2024-09-11T07:55:45.019 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.332+0000 7fafb9aa4640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fafb4076620 0x7fafb4074b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:53800/0 (socket says 172.21.15.102:53800) 2024-09-11T07:55:45.019 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.332+0000 7fafb9aa4640 1 -- 172.21.15.102:0/239078342 learned_addr learned my addr 172.21.15.102:0/239078342 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:55:45.019 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.333+0000 7fafb8aa2640 1 -- 172.21.15.102:0/239078342 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1531882130 0 0) 0x7fafb4075250 con 0x7fafb4076620 2024-09-11T07:55:45.020 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.333+0000 7fafb8aa2640 1 -- 172.21.15.102:0/239078342 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafa00034e0 con 0x7fafb4076620 2024-09-11T07:55:45.020 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.333+0000 7fafb8aa2640 1 -- 172.21.15.102:0/239078342 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1355003289 0 0) 0x7fafa00034e0 con 0x7fafb4076620 2024-09-11T07:55:45.020 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.333+0000 7fafb8aa2640 1 -- 172.21.15.102:0/239078342 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb4103ec0 con 0x7fafb4076620 2024-09-11T07:55:45.020 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.333+0000 7fafb8aa2640 1 -- 172.21.15.102:0/239078342 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fafb0001590 con 0x7fafb4076620 2024-09-11T07:55:45.020 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.333+0000 7fafb8aa2640 1 -- 172.21.15.102:0/239078342 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7fafb0001cb0 con 0x7fafb4076620 2024-09-11T07:55:45.020 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.333+0000 7fafb8aa2640 1 -- 172.21.15.102:0/239078342 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fafb0005670 con 0x7fafb4076620 2024-09-11T07:55:45.020 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.334+0000 7fafbbd08640 1 -- 172.21.15.102:0/239078342 >> v1:172.21.15.102:6789/0 conn(0x7fafb4076620 legacy=0x7fafb4074b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:55:45.020 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.334+0000 7fafbbd08640 1 -- 172.21.15.102:0/239078342 shutdown_connections 2024-09-11T07:55:45.020 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.334+0000 7fafbbd08640 1 -- 172.21.15.102:0/239078342 >> 172.21.15.102:0/239078342 conn(0x7fafb40fbba0 msgr2=0x7fafb40fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:55:45.020 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.334+0000 7fafbbd08640 1 -- 172.21.15.102:0/239078342 shutdown_connections 2024-09-11T07:55:45.020 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.334+0000 7fafbbd08640 1 -- 172.21.15.102:0/239078342 wait complete. 2024-09-11T07:55:45.020 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.334+0000 7fafbbd08640 1 Processor -- start 2024-09-11T07:55:45.020 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.334+0000 7fafbbd08640 1 -- start start 2024-09-11T07:55:45.020 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.334+0000 7fafbbd08640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fafb4115960 con 0x7fafb4076620 2024-09-11T07:55:45.021 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.335+0000 7fafb9aa4640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fafb4076620 0x7fafb4115250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:53806/0 (socket says 172.21.15.102:53806) 2024-09-11T07:55:45.021 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.335+0000 7fafb9aa4640 1 -- 172.21.15.102:0/2864057001 learned_addr learned my addr 172.21.15.102:0/2864057001 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:55:45.021 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.335+0000 7fafaa7fc640 1 -- 172.21.15.102:0/2864057001 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 36245489 0 0) 0x7fafb4115960 con 0x7fafb4076620 2024-09-11T07:55:45.021 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.335+0000 7fafaa7fc640 1 -- 172.21.15.102:0/2864057001 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafa00053f0 con 0x7fafb4076620 2024-09-11T07:55:45.021 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.335+0000 7fafaa7fc640 1 -- 172.21.15.102:0/2864057001 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3511107415 0 0) 0x7fafa00053f0 con 0x7fafb4076620 2024-09-11T07:55:45.021 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.335+0000 7fafaa7fc640 1 -- 172.21.15.102:0/2864057001 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fafb4115960 con 0x7fafb4076620 2024-09-11T07:55:45.021 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.335+0000 7fafaa7fc640 1 -- 172.21.15.102:0/2864057001 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fafb00047a0 con 0x7fafb4076620 2024-09-11T07:55:45.021 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.335+0000 7fafaa7fc640 1 -- 172.21.15.102:0/2864057001 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2375097445 0 0) 0x7fafb4115960 con 0x7fafb4076620 2024-09-11T07:55:45.021 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.335+0000 7fafaa7fc640 1 -- 172.21.15.102:0/2864057001 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb4115af0 con 0x7fafb4076620 2024-09-11T07:55:45.021 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.335+0000 7fafbbd08640 1 -- 172.21.15.102:0/2864057001 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb4115dc0 con 0x7fafb4076620 2024-09-11T07:55:45.021 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.335+0000 7fafbbd08640 1 -- 172.21.15.102:0/2864057001 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb41198a0 con 0x7fafb4076620 2024-09-11T07:55:45.021 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.335+0000 7fafaa7fc640 1 -- 172.21.15.102:0/2864057001 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7fafb0004ba0 con 0x7fafb4076620 2024-09-11T07:55:45.021 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.335+0000 7fafaa7fc640 1 -- 172.21.15.102:0/2864057001 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fafb0006030 con 0x7fafb4076620 2024-09-11T07:55:45.022 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.335+0000 7fafaa7fc640 1 -- 172.21.15.102:0/2864057001 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7fafb0006510 con 0x7fafb4076620 2024-09-11T07:55:45.022 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.335+0000 7fafaa7fc640 1 -- 172.21.15.102:0/2864057001 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 1638648040 0 0) 0x7fafb0007580 con 0x7fafb4076620 2024-09-11T07:55:45.022 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.336+0000 7fafbbd08640 1 -- 172.21.15.102:0/2864057001 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf98005040 con 0x7fafb4076620 2024-09-11T07:55:45.022 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.338+0000 7fafaa7fc640 1 -- 172.21.15.102:0/2864057001 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (unknown 1092875540 0 1542132831) 0x7fafb0004d10 con 0x7fafb4076620 2024-09-11T07:55:45.022 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.394+0000 7fafbbd08640 1 -- 172.21.15.102:0/2864057001 --> v1:172.21.15.102:6789/0 -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7faf98005870 con 0x7fafb4076620 2024-09-11T07:55:45.022 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.396+0000 7fafaa7fc640 1 -- 172.21.15.102:0/2864057001 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (unknown 534127980 0 0) 0x7fafb00059b0 con 0x7fafb4076620 2024-09-11T07:55:45.022 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.397+0000 7fafbbd08640 1 -- 172.21.15.102:0/2864057001 >> v1:172.21.15.102:6789/0 conn(0x7fafb4076620 legacy=0x7fafb4115250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:55:45.022 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.397+0000 7fafbbd08640 1 -- 172.21.15.102:0/2864057001 shutdown_connections 2024-09-11T07:55:45.022 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.397+0000 7fafbbd08640 1 -- 172.21.15.102:0/2864057001 >> 172.21.15.102:0/2864057001 conn(0x7fafb40fbba0 msgr2=0x7fafb40fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:55:45.022 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.397+0000 7fafbbd08640 1 -- 172.21.15.102:0/2864057001 shutdown_connections 2024-09-11T07:55:45.022 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:38.397+0000 7fafbbd08640 1 -- 172.21.15.102:0/2864057001 wait complete. 2024-09-11T07:55:45.022 INFO:teuthology.orchestra.run.smithi102.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-11T07:55:45.023 INFO:teuthology.orchestra.run.smithi102.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:55:45.023 INFO:teuthology.orchestra.run.smithi102.stdout:Creating mgr... 2024-09-11T07:55:45.023 INFO:teuthology.orchestra.run.smithi102.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-11T07:55:45.023 INFO:teuthology.orchestra.run.smithi102.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-11T07:55:45.261 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-2051892c-7013-11ef-bcea-c7b262605968@mgr.y 2024-09-11T07:55:45.261 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Failed to reset failed state of unit ceph-2051892c-7013-11ef-bcea-c7b262605968@mgr.y.service: Unit ceph-2051892c-7013-11ef-bcea-c7b262605968@mgr.y.service not loaded. 2024-09-11T07:55:45.615 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-2051892c-7013-11ef-bcea-c7b262605968.target.wants/ceph-2051892c-7013-11ef-bcea-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-2051892c-7013-11ef-bcea-c7b262605968@.service. 2024-09-11T07:55:45.896 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:45 smithi102 systemd[1]: Starting Ceph mgr.y for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T07:55:46.350 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:46 smithi102 podman[30085]: 2024-09-11 07:55:45.923468248 +0000 UTC m=+0.015619960 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:55:46.350 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:46 smithi102 podman[30085]: 2024-09-11 07:55:46.020964158 +0000 UTC m=+0.113115840 container create 5fd918c4c3f4558db2eb245fcc2d181441b1ed6dbe0bc8bbe91d74f8f4044d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:55:46.350 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:46 smithi102 podman[30085]: 2024-09-11 07:55:46.231493051 +0000 UTC m=+0.323644733 container init 5fd918c4c3f4558db2eb245fcc2d181441b1ed6dbe0bc8bbe91d74f8f4044d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-09-11T07:55:46.350 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:46 smithi102 podman[30085]: 2024-09-11 07:55:46.237036758 +0000 UTC m=+0.329188439 container start 5fd918c4c3f4558db2eb245fcc2d181441b1ed6dbe0bc8bbe91d74f8f4044d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:55:46.350 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:46 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:46.348+0000 7f3e7f863ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:55:46.373 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T07:55:46.373 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout disabled 2024-09-11T07:55:46.382 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T07:55:46.383 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout inactive 2024-09-11T07:55:46.383 INFO:teuthology.orchestra.run.smithi102.stdout:firewalld.service is not enabled 2024-09-11T07:55:46.383 INFO:teuthology.orchestra.run.smithi102.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T07:55:46.394 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T07:55:46.394 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout disabled 2024-09-11T07:55:46.403 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T07:55:46.403 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout inactive 2024-09-11T07:55:46.403 INFO:teuthology.orchestra.run.smithi102.stdout:firewalld.service is not enabled 2024-09-11T07:55:46.404 INFO:teuthology.orchestra.run.smithi102.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-11T07:55:46.404 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for mgr to start... 2024-09-11T07:55:46.404 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for mgr... 2024-09-11T07:55:46.611 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:46 smithi102 bash[30085]: 5fd918c4c3f4558db2eb245fcc2d181441b1ed6dbe0bc8bbe91d74f8f4044d96 2024-09-11T07:55:46.612 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:46 smithi102 systemd[1]: Started Ceph mgr.y for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T07:55:46.612 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:46 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:46.415+0000 7f3e7f863ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:55:46.612 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:46 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:46.481+0000 7f3e7f863ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:55:46.612 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:46 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:46.610+0000 7f3e7f863ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:55:46.861 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:46 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:46.755+0000 7f3e7f863ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:55:47.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2896565483' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T07:55:47.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:47 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:47.027+0000 7f3e7f863ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:55:47.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:47 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:47.088+0000 7f3e7f863ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:55:47.530 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:47 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:47.251+0000 7f3e7f863ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:55:47.535 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-09-11T07:55:47.535 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:55:47.535 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "fsid": "2051892c-7013-11ef-bcea-c7b262605968", 2024-09-11T07:55:47.535 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T07:55:47.535 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T07:55:47.535 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T07:55:47.535 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T07:55:47.535 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:55:47.535 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T07:55:47.535 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T07:55:47.535 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 0 2024-09-11T07:55:47.535 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:55:47.535 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T07:55:47.536 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T07:55:47.536 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:55:47.536 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-11T07:55:47.536 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T07:55:47.536 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:55:47.536 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T07:55:47.536 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T07:55:47.536 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:55:47.536 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T07:55:47.536 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:55:47.536 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T07:55:47.536 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T07:55:47.536 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T07:55:47.544 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T07:55:47.544 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T07:55:47.544 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T07:55:47.544 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:55:47.544 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T07:55:47.544 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T07:55:47.544 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T07:55:47.544 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T07:55:47.544 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T07:55:47.545 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T07:55:47.545 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T07:55:47.545 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T07:55:47.545 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T07:55:47.545 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:55:47.545 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T07:55:47.545 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:55:47.545 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T07:55:47.545 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T07:55:47.546 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:55:47.546 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T07:55:47.546 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T07:55:47.546 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T07:55:47.546 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T07:55:47.546 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T07:55:47.546 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T07:55:47.546 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T07:55:47.546 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:55:47.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:55:47.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:55:47.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T07:55:47.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:55:47.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T07:55:31.242330+0000", 2024-09-11T07:55:47.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:55:47.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:55:47.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T07:55:47.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:55:47.547 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.853+0000 7ff685c8a640 1 Processor -- start 2024-09-11T07:55:47.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.854+0000 7ff685c8a640 1 -- start start 2024-09-11T07:55:47.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.854+0000 7ff685c8a640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff680076120 con 0x7ff680075d10 2024-09-11T07:55:47.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.854+0000 7ff67f7fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ff680075d10 0x7ff680074290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:53824/0 (socket says 172.21.15.102:53824) 2024-09-11T07:55:47.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.854+0000 7ff67f7fe640 1 -- 172.21.15.102:0/2040069093 learned_addr learned my addr 172.21.15.102:0/2040069093 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:55:47.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.854+0000 7ff67e7fc640 1 -- 172.21.15.102:0/2040069093 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4117842082 0 0) 0x7ff680076120 con 0x7ff680075d10 2024-09-11T07:55:47.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.854+0000 7ff67e7fc640 1 -- 172.21.15.102:0/2040069093 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6680034e0 con 0x7ff680075d10 2024-09-11T07:55:47.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.854+0000 7ff67e7fc640 1 -- 172.21.15.102:0/2040069093 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1072832098 0 0) 0x7ff6680034e0 con 0x7ff680075d10 2024-09-11T07:55:47.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.854+0000 7ff67e7fc640 1 -- 172.21.15.102:0/2040069093 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6800749a0 con 0x7ff680075d10 2024-09-11T07:55:47.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.854+0000 7ff67e7fc640 1 -- 172.21.15.102:0/2040069093 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7ff670001590 con 0x7ff680075d10 2024-09-11T07:55:47.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.855+0000 7ff67e7fc640 1 -- 172.21.15.102:0/2040069093 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7ff670001be0 con 0x7ff680075d10 2024-09-11T07:55:47.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.855+0000 7ff67e7fc640 1 -- 172.21.15.102:0/2040069093 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7ff6700055b0 con 0x7ff680075d10 2024-09-11T07:55:47.548 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.855+0000 7ff685c8a640 1 -- 172.21.15.102:0/2040069093 >> v1:172.21.15.102:6789/0 conn(0x7ff680075d10 legacy=0x7ff680074290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:55:47.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.855+0000 7ff685c8a640 1 -- 172.21.15.102:0/2040069093 shutdown_connections 2024-09-11T07:55:47.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.855+0000 7ff685c8a640 1 -- 172.21.15.102:0/2040069093 >> 172.21.15.102:0/2040069093 conn(0x7ff6800fbb00 msgr2=0x7ff6800fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:55:47.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.855+0000 7ff685c8a640 1 -- 172.21.15.102:0/2040069093 shutdown_connections 2024-09-11T07:55:47.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.855+0000 7ff685c8a640 1 -- 172.21.15.102:0/2040069093 wait complete. 2024-09-11T07:55:47.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.856+0000 7ff685c8a640 1 Processor -- start 2024-09-11T07:55:47.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.856+0000 7ff685c8a640 1 -- start start 2024-09-11T07:55:47.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.856+0000 7ff685c8a640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff6801115d0 con 0x7ff680075d10 2024-09-11T07:55:47.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.856+0000 7ff67f7fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ff680075d10 0x7ff680110ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:53836/0 (socket says 172.21.15.102:53836) 2024-09-11T07:55:47.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.856+0000 7ff67f7fe640 1 -- 172.21.15.102:0/2896565483 learned_addr learned my addr 172.21.15.102:0/2896565483 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:55:47.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.857+0000 7ff667fff640 1 -- 172.21.15.102:0/2896565483 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2225860270 0 0) 0x7ff6801115d0 con 0x7ff680075d10 2024-09-11T07:55:47.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.857+0000 7ff667fff640 1 -- 172.21.15.102:0/2896565483 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff668005440 con 0x7ff680075d10 2024-09-11T07:55:47.549 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.857+0000 7ff667fff640 1 -- 172.21.15.102:0/2896565483 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1805855383 0 0) 0x7ff668005440 con 0x7ff680075d10 2024-09-11T07:55:47.550 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.857+0000 7ff667fff640 1 -- 172.21.15.102:0/2896565483 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff6801115d0 con 0x7ff680075d10 2024-09-11T07:55:47.550 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.857+0000 7ff667fff640 1 -- 172.21.15.102:0/2896565483 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7ff670001d50 con 0x7ff680075d10 2024-09-11T07:55:47.550 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.857+0000 7ff667fff640 1 -- 172.21.15.102:0/2896565483 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 600125197 0 0) 0x7ff6801115d0 con 0x7ff680075d10 2024-09-11T07:55:47.550 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.857+0000 7ff667fff640 1 -- 172.21.15.102:0/2896565483 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff680111760 con 0x7ff680075d10 2024-09-11T07:55:47.550 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.857+0000 7ff685c8a640 1 -- 172.21.15.102:0/2896565483 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff680111a30 con 0x7ff680075d10 2024-09-11T07:55:47.550 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.857+0000 7ff685c8a640 1 -- 172.21.15.102:0/2896565483 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff680115510 con 0x7ff680075d10 2024-09-11T07:55:47.550 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.857+0000 7ff667fff640 1 -- 172.21.15.102:0/2896565483 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7ff6700017e0 con 0x7ff680075d10 2024-09-11T07:55:47.550 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.857+0000 7ff667fff640 1 -- 172.21.15.102:0/2896565483 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7ff670005f20 con 0x7ff680075d10 2024-09-11T07:55:47.550 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.857+0000 7ff667fff640 1 -- 172.21.15.102:0/2896565483 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7ff6700063c0 con 0x7ff680075d10 2024-09-11T07:55:47.550 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.857+0000 7ff667fff640 1 -- 172.21.15.102:0/2896565483 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 1638648040 0 0) 0x7ff670007400 con 0x7ff680075d10 2024-09-11T07:55:47.550 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.858+0000 7ff685c8a640 1 -- 172.21.15.102:0/2896565483 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff66c005040 con 0x7ff680075d10 2024-09-11T07:55:47.550 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.860+0000 7ff667fff640 1 -- 172.21.15.102:0/2896565483 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (unknown 1092875540 0 1542132831) 0x7ff670001950 con 0x7ff680075d10 2024-09-11T07:55:47.550 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.919+0000 7ff685c8a640 1 -- 172.21.15.102:0/2896565483 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff66c0052f0 con 0x7ff680075d10 2024-09-11T07:55:47.551 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.919+0000 7ff667fff640 1 -- 172.21.15.102:0/2896565483 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1241 (unknown 4201413639 0 2455486942) 0x7ff6700058e0 con 0x7ff680075d10 2024-09-11T07:55:47.551 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.920+0000 7ff685c8a640 1 -- 172.21.15.102:0/2896565483 >> v1:172.21.15.102:6789/0 conn(0x7ff680075d10 legacy=0x7ff680110ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:55:47.551 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.920+0000 7ff685c8a640 1 -- 172.21.15.102:0/2896565483 shutdown_connections 2024-09-11T07:55:47.551 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.920+0000 7ff685c8a640 1 -- 172.21.15.102:0/2896565483 >> 172.21.15.102:0/2896565483 conn(0x7ff6800fbb00 msgr2=0x7ff6801053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:55:47.551 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.920+0000 7ff685c8a640 1 -- 172.21.15.102:0/2896565483 shutdown_connections 2024-09-11T07:55:47.551 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:46.920+0000 7ff685c8a640 1 -- 172.21.15.102:0/2896565483 wait complete. 2024-09-11T07:55:47.551 INFO:teuthology.orchestra.run.smithi102.stdout:mgr not available, waiting (1/15)... 2024-09-11T07:55:48.395 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:48 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:48.097+0000 7f3e7f863ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:55:48.396 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:48 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:48.161+0000 7f3e7f863ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:55:49.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:48 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:48.866+0000 7f3e7f863ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:55:49.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:48 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:48.975+0000 7f3e7f863ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:55:49.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:49.094+0000 7f3e7f863ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:55:49.535 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:49.154+0000 7f3e7f863ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:55:49.536 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:49.278+0000 7f3e7f863ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:55:49.536 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:49.344+0000 7f3e7f863ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:55:49.536 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:49.401+0000 7f3e7f863ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:55:49.536 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:49.534+0000 7f3e7f863ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:55:49.807 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:49.602+0000 7f3e7f863ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:55:49.807 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:49.667+0000 7f3e7f863ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:55:49.807 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:49.749+0000 7f3e7f863ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:55:50.072 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:49.810+0000 7f3e7f863ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:55:50.072 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:49.870+0000 7f3e7f863ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:55:50.369 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:50 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/115872200' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T07:55:50.683 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-09-11T07:55:50.683 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:55:50.684 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "fsid": "2051892c-7013-11ef-bcea-c7b262605968", 2024-09-11T07:55:50.684 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T07:55:50.684 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T07:55:50.684 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T07:55:50.684 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T07:55:50.684 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:55:50.684 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T07:55:50.684 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T07:55:50.684 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 0 2024-09-11T07:55:50.684 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:55:50.684 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T07:55:50.684 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T07:55:50.684 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:55:50.684 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-09-11T07:55:50.685 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T07:55:50.685 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:55:50.685 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T07:55:50.685 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T07:55:50.685 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:55:50.685 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T07:55:50.685 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:55:50.685 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T07:55:50.685 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T07:55:50.685 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T07:55:50.685 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T07:55:50.685 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T07:55:50.685 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T07:55:50.692 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:55:50.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T07:55:50.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T07:55:50.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T07:55:50.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T07:55:50.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T07:55:50.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T07:55:50.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T07:55:50.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T07:55:50.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T07:55:50.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:55:50.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T07:55:50.693 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:55:50.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T07:55:50.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T07:55:50.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:55:50.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T07:55:50.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T07:55:50.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T07:55:50.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T07:55:50.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T07:55:50.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T07:55:50.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T07:55:50.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:55:50.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:55:50.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:55:50.694 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T07:55:50.695 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:55:50.695 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T07:55:31.242330+0000", 2024-09-11T07:55:50.695 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:55:50.695 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:55:50.695 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T07:55:50.695 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:55:50.695 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.006+0000 7fb3f4af2640 1 Processor -- start 2024-09-11T07:55:50.695 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.007+0000 7fb3f4af2640 1 -- start start 2024-09-11T07:55:50.695 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.007+0000 7fb3f4af2640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb3f0100f80 con 0x7fb3f0100460 2024-09-11T07:55:50.695 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.007+0000 7fb3ee59c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb3f0100460 0x7fb3f0100870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45366/0 (socket says 172.21.15.102:45366) 2024-09-11T07:55:50.695 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.007+0000 7fb3ee59c640 1 -- 172.21.15.102:0/3217797458 learned_addr learned my addr 172.21.15.102:0/3217797458 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:55:50.695 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.007+0000 7fb3ed59a640 1 -- 172.21.15.102:0/3217797458 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235546146 0 0) 0x7fb3f0100f80 con 0x7fb3f0100460 2024-09-11T07:55:50.695 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.007+0000 7fb3ed59a640 1 -- 172.21.15.102:0/3217797458 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3d80034e0 con 0x7fb3f0100460 2024-09-11T07:55:50.696 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.008+0000 7fb3ed59a640 1 -- 172.21.15.102:0/3217797458 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2447369017 0 0) 0x7fb3d80034e0 con 0x7fb3f0100460 2024-09-11T07:55:50.696 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.008+0000 7fb3ed59a640 1 -- 172.21.15.102:0/3217797458 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f0105920 con 0x7fb3f0100460 2024-09-11T07:55:50.696 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.008+0000 7fb3ed59a640 1 -- 172.21.15.102:0/3217797458 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fb3e0001590 con 0x7fb3f0100460 2024-09-11T07:55:50.696 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.008+0000 7fb3ed59a640 1 -- 172.21.15.102:0/3217797458 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7fb3e0001cb0 con 0x7fb3f0100460 2024-09-11T07:55:50.696 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.008+0000 7fb3ed59a640 1 -- 172.21.15.102:0/3217797458 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fb3e0005670 con 0x7fb3f0100460 2024-09-11T07:55:50.696 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.008+0000 7fb3f4af2640 1 -- 172.21.15.102:0/3217797458 >> v1:172.21.15.102:6789/0 conn(0x7fb3f0100460 legacy=0x7fb3f0100870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:55:50.696 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.008+0000 7fb3f4af2640 1 -- 172.21.15.102:0/3217797458 shutdown_connections 2024-09-11T07:55:50.696 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.008+0000 7fb3f4af2640 1 -- 172.21.15.102:0/3217797458 >> 172.21.15.102:0/3217797458 conn(0x7fb3f00fbbb0 msgr2=0x7fb3f00fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:55:50.696 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.008+0000 7fb3f4af2640 1 -- 172.21.15.102:0/3217797458 shutdown_connections 2024-09-11T07:55:50.696 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.008+0000 7fb3f4af2640 1 -- 172.21.15.102:0/3217797458 wait complete. 2024-09-11T07:55:50.696 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.009+0000 7fb3f4af2640 1 Processor -- start 2024-09-11T07:55:50.696 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.009+0000 7fb3f4af2640 1 -- start start 2024-09-11T07:55:50.696 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.009+0000 7fb3f4af2640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb3f010f360 con 0x7fb3f0100460 2024-09-11T07:55:50.697 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.009+0000 7fb3ee59c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb3f0100460 0x7fb3f010ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45370/0 (socket says 172.21.15.102:45370) 2024-09-11T07:55:50.697 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.009+0000 7fb3ee59c640 1 -- 172.21.15.102:0/115872200 learned_addr learned my addr 172.21.15.102:0/115872200 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:55:50.697 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.009+0000 7fb3deffd640 1 -- 172.21.15.102:0/115872200 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425831904 0 0) 0x7fb3f010f360 con 0x7fb3f0100460 2024-09-11T07:55:50.697 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.009+0000 7fb3deffd640 1 -- 172.21.15.102:0/115872200 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3d80053f0 con 0x7fb3f0100460 2024-09-11T07:55:50.697 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.010+0000 7fb3deffd640 1 -- 172.21.15.102:0/115872200 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3949661174 0 0) 0x7fb3d80053f0 con 0x7fb3f0100460 2024-09-11T07:55:50.697 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.010+0000 7fb3deffd640 1 -- 172.21.15.102:0/115872200 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb3f010f360 con 0x7fb3f0100460 2024-09-11T07:55:50.697 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.010+0000 7fb3deffd640 1 -- 172.21.15.102:0/115872200 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fb3e00047a0 con 0x7fb3f0100460 2024-09-11T07:55:50.697 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.010+0000 7fb3deffd640 1 -- 172.21.15.102:0/115872200 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1459964264 0 0) 0x7fb3f010f360 con 0x7fb3f0100460 2024-09-11T07:55:50.697 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.010+0000 7fb3deffd640 1 -- 172.21.15.102:0/115872200 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f010f4f0 con 0x7fb3f0100460 2024-09-11T07:55:50.697 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.010+0000 7fb3f4af2640 1 -- 172.21.15.102:0/115872200 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3f010f7c0 con 0x7fb3f0100460 2024-09-11T07:55:50.697 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.010+0000 7fb3f4af2640 1 -- 172.21.15.102:0/115872200 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3f0113250 con 0x7fb3f0100460 2024-09-11T07:55:50.697 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.010+0000 7fb3deffd640 1 -- 172.21.15.102:0/115872200 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7fb3e00049e0 con 0x7fb3f0100460 2024-09-11T07:55:50.697 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.010+0000 7fb3deffd640 1 -- 172.21.15.102:0/115872200 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fb3e0006050 con 0x7fb3f0100460 2024-09-11T07:55:50.698 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.010+0000 7fb3deffd640 1 -- 172.21.15.102:0/115872200 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7fb3e00064f0 con 0x7fb3f0100460 2024-09-11T07:55:50.698 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.010+0000 7fb3deffd640 1 -- 172.21.15.102:0/115872200 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 1638648040 0 0) 0x7fb3e0007560 con 0x7fb3f0100460 2024-09-11T07:55:50.698 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.011+0000 7fb3f4af2640 1 -- 172.21.15.102:0/115872200 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d0005040 con 0x7fb3f0100460 2024-09-11T07:55:50.698 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.013+0000 7fb3deffd640 1 -- 172.21.15.102:0/115872200 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (unknown 1092875540 0 1542132831) 0x7fb3e0004ba0 con 0x7fb3f0100460 2024-09-11T07:55:50.698 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.070+0000 7fb3f4af2640 1 -- 172.21.15.102:0/115872200 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb3d00052f0 con 0x7fb3f0100460 2024-09-11T07:55:50.698 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.071+0000 7fb3deffd640 1 -- 172.21.15.102:0/115872200 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1241 (unknown 4201413639 0 2167766510) 0x7fb3e0004de0 con 0x7fb3f0100460 2024-09-11T07:55:50.698 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.072+0000 7fb3f4af2640 1 -- 172.21.15.102:0/115872200 >> v1:172.21.15.102:6789/0 conn(0x7fb3f0100460 legacy=0x7fb3f010ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:55:50.698 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.072+0000 7fb3f4af2640 1 -- 172.21.15.102:0/115872200 shutdown_connections 2024-09-11T07:55:50.698 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.072+0000 7fb3f4af2640 1 -- 172.21.15.102:0/115872200 >> 172.21.15.102:0/115872200 conn(0x7fb3f00fbbb0 msgr2=0x7fb3f0105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:55:50.698 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.072+0000 7fb3f4af2640 1 -- 172.21.15.102:0/115872200 shutdown_connections 2024-09-11T07:55:50.698 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:50.072+0000 7fb3f4af2640 1 -- 172.21.15.102:0/115872200 wait complete. 2024-09-11T07:55:50.698 INFO:teuthology.orchestra.run.smithi102.stdout:mgr not available, waiting (2/15)... 2024-09-11T07:55:50.895 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:50 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:50.634+0000 7f3e7f863ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:55:51.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:51 smithi102 ceph-mon[29849]: Activating manager daemon y 2024-09-11T07:55:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:51 smithi102 ceph-mon[29849]: mgrmap e2: y(active, starting, since 0.0027261s) 2024-09-11T07:55:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:51 smithi102 ceph-mon[29849]: from='mgr.14100 v1:172.21.15.102:0/1172129757' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:55:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:51 smithi102 ceph-mon[29849]: from='mgr.14100 v1:172.21.15.102:0/1172129757' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:55:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:51 smithi102 ceph-mon[29849]: from='mgr.14100 v1:172.21.15.102:0/1172129757' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:55:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:51 smithi102 ceph-mon[29849]: from='mgr.14100 v1:172.21.15.102:0/1172129757' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:55:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:51 smithi102 ceph-mon[29849]: from='mgr.14100 v1:172.21.15.102:0/1172129757' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:55:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:51 smithi102 ceph-mon[29849]: from='mgr.14100 v1:172.21.15.102:0/1172129757' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:55:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:51 smithi102 ceph-mon[29849]: from='mgr.14100 v1:172.21.15.102:0/1172129757' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:55:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:51 smithi102 ceph-mon[29849]: from='mgr.14100 v1:172.21.15.102:0/1172129757' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:55:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:51 smithi102 ceph-mon[29849]: Manager daemon y is now available 2024-09-11T07:55:51.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:51 smithi102 ceph-mon[29849]: from='mgr.14100 v1:172.21.15.102:0/1172129757' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:55:51.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:51 smithi102 ceph-mon[29849]: from='mgr.14100 v1:172.21.15.102:0/1172129757' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:55:51.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:51 smithi102 ceph-mon[29849]: from='mgr.14100 v1:172.21.15.102:0/1172129757' entity='mgr.y' 2024-09-11T07:55:51.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:51 smithi102 ceph-mon[29849]: from='mgr.14100 v1:172.21.15.102:0/1172129757' entity='mgr.y' 2024-09-11T07:55:51.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:51 smithi102 ceph-mon[29849]: from='mgr.14100 v1:172.21.15.102:0/1172129757' entity='mgr.y' 2024-09-11T07:55:52.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:52 smithi102 ceph-mon[29849]: mgrmap e3: y(active, since 1.00577s) 2024-09-11T07:55:53.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:53 smithi102 ceph-mon[29849]: mgrmap e4: y(active, since 2s) 2024-09-11T07:55:54.892 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:54 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/906094533' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T07:55:54.892 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-09-11T07:55:54.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:55:54.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "fsid": "2051892c-7013-11ef-bcea-c7b262605968", 2024-09-11T07:55:54.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T07:55:54.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T07:55:54.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T07:55:54.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T07:55:54.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:55:54.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T07:55:54.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T07:55:54.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 0 2024-09-11T07:55:54.895 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:55:54.896 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T07:55:54.896 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T07:55:54.896 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:55:54.896 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "quorum_age": 18, 2024-09-11T07:55:54.896 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T07:55:54.896 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:55:54.896 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T07:55:54.896 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T07:55:54.896 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:55:54.896 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T07:55:54.896 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:55:54.896 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T07:55:54.896 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T07:55:54.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T07:55:54.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T07:55:54.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T07:55:54.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T07:55:54.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:55:54.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T07:55:54.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T07:55:54.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T07:55:54.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T07:55:54.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T07:55:54.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T07:55:54.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T07:55:54.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T07:55:54.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T07:55:54.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:55:54.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T07:55:54.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:55:54.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T07:55:54.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T07:55:54.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:55:54.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T07:55:54.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T07:55:54.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T07:55:54.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T07:55:54.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T07:55:54.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T07:55:54.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T07:55:54.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:55:54.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:55:54.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:55:54.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T07:55:54.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:55:54.906 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T07:55:31.242330+0000", 2024-09-11T07:55:54.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:55:54.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:55:54.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T07:55:54.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:55:54.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.138+0000 7fcb88980640 1 Processor -- start 2024-09-11T07:55:54.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.138+0000 7fcb88980640 1 -- start start 2024-09-11T07:55:54.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.138+0000 7fcb88980640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcb841046c0 con 0x7fcb84073c10 2024-09-11T07:55:54.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.139+0000 7fcb8259c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fcb84073c10 0x7fcb84102fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45432/0 (socket says 172.21.15.102:45432) 2024-09-11T07:55:54.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.139+0000 7fcb8259c640 1 -- 172.21.15.102:0/2894444087 learned_addr learned my addr 172.21.15.102:0/2894444087 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:55:54.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.139+0000 7fcb8159a640 1 -- 172.21.15.102:0/2894444087 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1397708598 0 0) 0x7fcb841046c0 con 0x7fcb84073c10 2024-09-11T07:55:54.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.139+0000 7fcb8159a640 1 -- 172.21.15.102:0/2894444087 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb700034e0 con 0x7fcb84073c10 2024-09-11T07:55:54.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.139+0000 7fcb8159a640 1 -- 172.21.15.102:0/2894444087 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 197379452 0 0) 0x7fcb700034e0 con 0x7fcb84073c10 2024-09-11T07:55:54.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.139+0000 7fcb8159a640 1 -- 172.21.15.102:0/2894444087 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb84105860 con 0x7fcb84073c10 2024-09-11T07:55:54.907 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.139+0000 7fcb8159a640 1 -- 172.21.15.102:0/2894444087 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fcb7c001590 con 0x7fcb84073c10 2024-09-11T07:55:54.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.139+0000 7fcb8159a640 1 -- 172.21.15.102:0/2894444087 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7fcb7c001cb0 con 0x7fcb84073c10 2024-09-11T07:55:54.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.139+0000 7fcb8159a640 1 -- 172.21.15.102:0/2894444087 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fcb7c005670 con 0x7fcb84073c10 2024-09-11T07:55:54.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.140+0000 7fcb88980640 1 -- 172.21.15.102:0/2894444087 >> v1:172.21.15.102:6789/0 conn(0x7fcb84073c10 legacy=0x7fcb84102fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:55:54.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.140+0000 7fcb88980640 1 -- 172.21.15.102:0/2894444087 shutdown_connections 2024-09-11T07:55:54.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.140+0000 7fcb88980640 1 -- 172.21.15.102:0/2894444087 >> 172.21.15.102:0/2894444087 conn(0x7fcb840fa9d0 msgr2=0x7fcb840fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:55:54.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.140+0000 7fcb88980640 1 -- 172.21.15.102:0/2894444087 shutdown_connections 2024-09-11T07:55:54.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.140+0000 7fcb88980640 1 -- 172.21.15.102:0/2894444087 wait complete. 2024-09-11T07:55:54.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.141+0000 7fcb88980640 1 Processor -- start 2024-09-11T07:55:54.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.141+0000 7fcb88980640 1 -- start start 2024-09-11T07:55:54.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.141+0000 7fcb88980640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcb8410f330 con 0x7fcb84073c10 2024-09-11T07:55:54.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.141+0000 7fcb8259c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fcb84073c10 0x7fcb8410ec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45444/0 (socket says 172.21.15.102:45444) 2024-09-11T07:55:54.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.141+0000 7fcb8259c640 1 -- 172.21.15.102:0/906094533 learned_addr learned my addr 172.21.15.102:0/906094533 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:55:54.908 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.141+0000 7fcb7affd640 1 -- 172.21.15.102:0/906094533 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567927315 0 0) 0x7fcb8410f330 con 0x7fcb84073c10 2024-09-11T07:55:54.909 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.141+0000 7fcb7affd640 1 -- 172.21.15.102:0/906094533 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb700053f0 con 0x7fcb84073c10 2024-09-11T07:55:54.909 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.141+0000 7fcb7affd640 1 -- 172.21.15.102:0/906094533 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2911638185 0 0) 0x7fcb700053f0 con 0x7fcb84073c10 2024-09-11T07:55:54.909 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.141+0000 7fcb7affd640 1 -- 172.21.15.102:0/906094533 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcb8410f330 con 0x7fcb84073c10 2024-09-11T07:55:54.909 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.141+0000 7fcb7affd640 1 -- 172.21.15.102:0/906094533 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fcb7c0047a0 con 0x7fcb84073c10 2024-09-11T07:55:54.909 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.142+0000 7fcb7affd640 1 -- 172.21.15.102:0/906094533 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4289739701 0 0) 0x7fcb8410f330 con 0x7fcb84073c10 2024-09-11T07:55:54.909 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.142+0000 7fcb7affd640 1 -- 172.21.15.102:0/906094533 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8410f4c0 con 0x7fcb84073c10 2024-09-11T07:55:54.909 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.142+0000 7fcb88980640 1 -- 172.21.15.102:0/906094533 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8410f790 con 0x7fcb84073c10 2024-09-11T07:55:54.909 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.142+0000 7fcb88980640 1 -- 172.21.15.102:0/906094533 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb84113270 con 0x7fcb84073c10 2024-09-11T07:55:54.909 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.142+0000 7fcb7affd640 1 -- 172.21.15.102:0/906094533 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7fcb7c004ba0 con 0x7fcb84073c10 2024-09-11T07:55:54.909 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.142+0000 7fcb7affd640 1 -- 172.21.15.102:0/906094533 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fcb7c006170 con 0x7fcb84073c10 2024-09-11T07:55:54.909 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.142+0000 7fcb7affd640 1 -- 172.21.15.102:0/906094533 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 4) v1 ==== 45446+0+0 (unknown 4184903468 0 0) 0x7fcb7c011480 con 0x7fcb84073c10 2024-09-11T07:55:54.909 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.142+0000 7fcb88980640 1 -- 172.21.15.102:0/906094533 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb84105350 con 0x7fcb84073c10 2024-09-11T07:55:54.910 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.143+0000 7fcb7affd640 1 -- 172.21.15.102:0/906094533 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 1638648040 0 0) 0x7fcb7c047920 con 0x7fcb84073c10 2024-09-11T07:55:54.910 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.147+0000 7fcb7affd640 1 -- 172.21.15.102:0/906094533 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fcb7c012b30 con 0x7fcb84073c10 2024-09-11T07:55:54.910 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.358+0000 7fcb88980640 1 -- 172.21.15.102:0/906094533 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fcb84113520 con 0x7fcb84073c10 2024-09-11T07:55:54.910 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.358+0000 7fcb7affd640 1 -- 172.21.15.102:0/906094533 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (unknown 4201413639 0 1736672054) 0x7fcb7c0164f0 con 0x7fcb84073c10 2024-09-11T07:55:54.910 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.360+0000 7fcb88980640 1 -- 172.21.15.102:0/906094533 >> v1:172.21.15.102:6800/3158655170 conn(0x7fcb7003a650 legacy=0x7fcb7003cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:55:54.910 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.360+0000 7fcb88980640 1 -- 172.21.15.102:0/906094533 >> v1:172.21.15.102:6789/0 conn(0x7fcb84073c10 legacy=0x7fcb8410ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:55:54.910 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.361+0000 7fcb88980640 1 -- 172.21.15.102:0/906094533 shutdown_connections 2024-09-11T07:55:54.910 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.361+0000 7fcb88980640 1 -- 172.21.15.102:0/906094533 >> 172.21.15.102:0/906094533 conn(0x7fcb840fa9d0 msgr2=0x7fcb841028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:55:54.910 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.361+0000 7fcb88980640 1 -- 172.21.15.102:0/906094533 shutdown_connections 2024-09-11T07:55:54.910 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:54.361+0000 7fcb88980640 1 -- 172.21.15.102:0/906094533 wait complete. 2024-09-11T07:55:54.910 INFO:teuthology.orchestra.run.smithi102.stdout:mgr is available 2024-09-11T07:55:55.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2466938639' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T07:55:56.096 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-09-11T07:55:56.096 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T07:55:56.096 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout fsid = 2051892c-7013-11ef-bcea-c7b262605968 2024-09-11T07:55:56.096 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T07:55:56.096 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T07:55:56.097 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T07:55:56.097 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T07:55:56.097 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T07:55:56.097 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-09-11T07:55:56.097 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T07:55:56.097 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T07:55:56.097 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 2024-09-11T07:55:56.097 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T07:55:56.097 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T07:55:56.097 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T07:55:56.097 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.325+0000 7f790dadb640 1 Processor -- start 2024-09-11T07:55:56.097 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.325+0000 7f790dadb640 1 -- start start 2024-09-11T07:55:56.097 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.326+0000 7f790dadb640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f79080ffd70 con 0x7f79080ff250 2024-09-11T07:55:56.098 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.326+0000 7f79077fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f79080ff250 0x7f79080ff660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45448/0 (socket says 172.21.15.102:45448) 2024-09-11T07:55:56.098 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.326+0000 7f79077fe640 1 -- 172.21.15.102:0/3493369674 learned_addr learned my addr 172.21.15.102:0/3493369674 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:55:56.098 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.326+0000 7f79067fc640 1 -- 172.21.15.102:0/3493369674 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 670476726 0 0) 0x7f79080ffd70 con 0x7f79080ff250 2024-09-11T07:55:56.098 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.326+0000 7f79067fc640 1 -- 172.21.15.102:0/3493369674 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78f40034e0 con 0x7f79080ff250 2024-09-11T07:55:56.098 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.326+0000 7f79067fc640 1 -- 172.21.15.102:0/3493369674 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1162816044 0 0) 0x7f78f40034e0 con 0x7f79080ff250 2024-09-11T07:55:56.098 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.326+0000 7f79067fc640 1 -- 172.21.15.102:0/3493369674 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7908104710 con 0x7f79080ff250 2024-09-11T07:55:56.098 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.326+0000 7f79067fc640 1 -- 172.21.15.102:0/3493369674 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f78f8001590 con 0x7f79080ff250 2024-09-11T07:55:56.098 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.326+0000 7f79067fc640 1 -- 172.21.15.102:0/3493369674 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7f78f8001cb0 con 0x7f79080ff250 2024-09-11T07:55:56.098 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.326+0000 7f79067fc640 1 -- 172.21.15.102:0/3493369674 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f78f8005670 con 0x7f79080ff250 2024-09-11T07:55:56.098 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.327+0000 7f790dadb640 1 -- 172.21.15.102:0/3493369674 >> v1:172.21.15.102:6789/0 conn(0x7f79080ff250 legacy=0x7f79080ff660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:55:56.098 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.327+0000 7f790dadb640 1 -- 172.21.15.102:0/3493369674 shutdown_connections 2024-09-11T07:55:56.098 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.327+0000 7f790dadb640 1 -- 172.21.15.102:0/3493369674 >> 172.21.15.102:0/3493369674 conn(0x7f79080fa9d0 msgr2=0x7f79080fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:55:56.098 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.327+0000 7f790dadb640 1 -- 172.21.15.102:0/3493369674 shutdown_connections 2024-09-11T07:55:56.099 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.327+0000 7f790dadb640 1 -- 172.21.15.102:0/3493369674 wait complete. 2024-09-11T07:55:56.099 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.327+0000 7f790dadb640 1 Processor -- start 2024-09-11T07:55:56.099 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.328+0000 7f790dadb640 1 -- start start 2024-09-11T07:55:56.099 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.328+0000 7f790dadb640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f79080fceb0 con 0x7f79080ff250 2024-09-11T07:55:56.099 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.328+0000 7f79077fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f79080ff250 0x7f79080fc7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45454/0 (socket says 172.21.15.102:45454) 2024-09-11T07:55:56.099 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.328+0000 7f79077fe640 1 -- 172.21.15.102:0/2466938639 learned_addr learned my addr 172.21.15.102:0/2466938639 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:55:56.099 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.328+0000 7f78fffff640 1 -- 172.21.15.102:0/2466938639 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929426008 0 0) 0x7f79080fceb0 con 0x7f79080ff250 2024-09-11T07:55:56.099 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.328+0000 7f78fffff640 1 -- 172.21.15.102:0/2466938639 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78f40053f0 con 0x7f79080ff250 2024-09-11T07:55:56.099 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.328+0000 7f78fffff640 1 -- 172.21.15.102:0/2466938639 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3592029543 0 0) 0x7f78f40053f0 con 0x7f79080ff250 2024-09-11T07:55:56.099 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.328+0000 7f78fffff640 1 -- 172.21.15.102:0/2466938639 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f79080fceb0 con 0x7f79080ff250 2024-09-11T07:55:56.099 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.328+0000 7f78fffff640 1 -- 172.21.15.102:0/2466938639 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f78f80047a0 con 0x7f79080ff250 2024-09-11T07:55:56.099 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.328+0000 7f78fffff640 1 -- 172.21.15.102:0/2466938639 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 87759582 0 0) 0x7f79080fceb0 con 0x7f79080ff250 2024-09-11T07:55:56.099 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.328+0000 7f78fffff640 1 -- 172.21.15.102:0/2466938639 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7908115d00 con 0x7f79080ff250 2024-09-11T07:55:56.100 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.328+0000 7f790dadb640 1 -- 172.21.15.102:0/2466938639 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7908114d50 con 0x7f79080ff250 2024-09-11T07:55:56.100 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.328+0000 7f790dadb640 1 -- 172.21.15.102:0/2466938639 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7908115220 con 0x7f79080ff250 2024-09-11T07:55:56.100 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.329+0000 7f78fffff640 1 -- 172.21.15.102:0/2466938639 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7f78f80049e0 con 0x7f79080ff250 2024-09-11T07:55:56.100 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.329+0000 7f78fffff640 1 -- 172.21.15.102:0/2466938639 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f78f8006070 con 0x7f79080ff250 2024-09-11T07:55:56.100 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.329+0000 7f78fffff640 1 -- 172.21.15.102:0/2466938639 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 4) v1 ==== 45446+0+0 (unknown 4184903468 0 0) 0x7f78f80113e0 con 0x7f79080ff250 2024-09-11T07:55:56.100 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.329+0000 7f790dadb640 1 -- 172.21.15.102:0/2466938639 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e8005040 con 0x7f79080ff250 2024-09-11T07:55:56.100 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.330+0000 7f78fffff640 1 -- 172.21.15.102:0/2466938639 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 1638648040 0 0) 0x7f78f8047930 con 0x7f79080ff250 2024-09-11T07:55:56.100 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.334+0000 7f78fffff640 1 -- 172.21.15.102:0/2466938639 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f78f8012bd0 con 0x7f79080ff250 2024-09-11T07:55:56.100 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.489+0000 7f790dadb640 1 -- 172.21.15.102:0/2466938639 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f78e8005870 con 0x7f79080ff250 2024-09-11T07:55:56.100 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.490+0000 7f78fffff640 1 -- 172.21.15.102:0/2466938639 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+318 (unknown 1187553405 0 2758924029) 0x7f78f8016550 con 0x7f79080ff250 2024-09-11T07:55:56.100 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.491+0000 7f790dadb640 1 -- 172.21.15.102:0/2466938639 >> v1:172.21.15.102:6800/3158655170 conn(0x7f78f403a5f0 legacy=0x7f78f403caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:55:56.100 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.491+0000 7f790dadb640 1 -- 172.21.15.102:0/2466938639 >> v1:172.21.15.102:6789/0 conn(0x7f79080ff250 legacy=0x7f79080fc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:55:56.100 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.491+0000 7f790dadb640 1 -- 172.21.15.102:0/2466938639 shutdown_connections 2024-09-11T07:55:56.100 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.491+0000 7f790dadb640 1 -- 172.21.15.102:0/2466938639 >> 172.21.15.102:0/2466938639 conn(0x7f79080fa9d0 msgr2=0x7f7908072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:55:56.101 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.492+0000 7f790dadb640 1 -- 172.21.15.102:0/2466938639 shutdown_connections 2024-09-11T07:55:56.101 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:55.492+0000 7f790dadb640 1 -- 172.21.15.102:0/2466938639 wait complete. 2024-09-11T07:55:56.101 INFO:teuthology.orchestra.run.smithi102.stdout:Enabling cephadm module... 2024-09-11T07:55:56.976 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:56 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1369898545' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-11T07:55:56.977 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:56 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ignoring --setuser ceph since I am not root 2024-09-11T07:55:56.977 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:56 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ignoring --setgroup ceph since I am not root 2024-09-11T07:55:56.977 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:56 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:56.857+0000 7f027519bec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:55:56.977 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:56 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:56.911+0000 7f027519bec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:55:56.977 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:56 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:56.974+0000 7f027519bec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:55:57.233 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:57 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:57.099+0000 7f027519bec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:55:57.233 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:57 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:57.232+0000 7f027519bec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:55:57.394 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.473+0000 7fd8a37d7640 1 Processor -- start 2024-09-11T07:55:57.394 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.474+0000 7fd8a37d7640 1 -- start start 2024-09-11T07:55:57.394 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.474+0000 7fd8a37d7640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd89c1046c0 con 0x7fd89c0744f0 2024-09-11T07:55:57.394 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.474+0000 7fd8a1573640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd89c0744f0 0x7fd89c102fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45460/0 (socket says 172.21.15.102:45460) 2024-09-11T07:55:57.394 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.474+0000 7fd8a1573640 1 -- 172.21.15.102:0/2549996365 learned_addr learned my addr 172.21.15.102:0/2549996365 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:55:57.394 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.474+0000 7fd88ffff640 1 -- 172.21.15.102:0/2549996365 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 172892889 0 0) 0x7fd89c1046c0 con 0x7fd89c0744f0 2024-09-11T07:55:57.394 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.474+0000 7fd88ffff640 1 -- 172.21.15.102:0/2549996365 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8880034e0 con 0x7fd89c0744f0 2024-09-11T07:55:57.394 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.474+0000 7fd88ffff640 1 -- 172.21.15.102:0/2549996365 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2629770516 0 0) 0x7fd8880034e0 con 0x7fd89c0744f0 2024-09-11T07:55:57.394 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.474+0000 7fd88ffff640 1 -- 172.21.15.102:0/2549996365 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c105860 con 0x7fd89c0744f0 2024-09-11T07:55:57.395 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.475+0000 7fd88ffff640 1 -- 172.21.15.102:0/2549996365 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fd898001590 con 0x7fd89c0744f0 2024-09-11T07:55:57.395 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.475+0000 7fd88ffff640 1 -- 172.21.15.102:0/2549996365 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7fd898001cb0 con 0x7fd89c0744f0 2024-09-11T07:55:57.395 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.475+0000 7fd88ffff640 1 -- 172.21.15.102:0/2549996365 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fd898005670 con 0x7fd89c0744f0 2024-09-11T07:55:57.395 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.475+0000 7fd8a37d7640 1 -- 172.21.15.102:0/2549996365 >> v1:172.21.15.102:6789/0 conn(0x7fd89c0744f0 legacy=0x7fd89c102fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:55:57.395 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.475+0000 7fd8a37d7640 1 -- 172.21.15.102:0/2549996365 shutdown_connections 2024-09-11T07:55:57.395 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.475+0000 7fd8a37d7640 1 -- 172.21.15.102:0/2549996365 >> 172.21.15.102:0/2549996365 conn(0x7fd89c0faa90 msgr2=0x7fd89c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:55:57.395 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.475+0000 7fd8a37d7640 1 -- 172.21.15.102:0/2549996365 shutdown_connections 2024-09-11T07:55:57.395 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.475+0000 7fd8a37d7640 1 -- 172.21.15.102:0/2549996365 wait complete. 2024-09-11T07:55:57.395 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.476+0000 7fd8a37d7640 1 Processor -- start 2024-09-11T07:55:57.395 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.476+0000 7fd8a37d7640 1 -- start start 2024-09-11T07:55:57.395 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.476+0000 7fd8a37d7640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd89c10f3b0 con 0x7fd89c0744f0 2024-09-11T07:55:57.395 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.476+0000 7fd8a1573640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd89c0744f0 0x7fd89c10eca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:45464/0 (socket says 172.21.15.102:45464) 2024-09-11T07:55:57.396 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.476+0000 7fd8a1573640 1 -- 172.21.15.102:0/1369898545 learned_addr learned my addr 172.21.15.102:0/1369898545 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:55:57.396 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.476+0000 7fd88dffb640 1 -- 172.21.15.102:0/1369898545 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1743595698 0 0) 0x7fd89c10f3b0 con 0x7fd89c0744f0 2024-09-11T07:55:57.396 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.476+0000 7fd88dffb640 1 -- 172.21.15.102:0/1369898545 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8880053f0 con 0x7fd89c0744f0 2024-09-11T07:55:57.396 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.476+0000 7fd88dffb640 1 -- 172.21.15.102:0/1369898545 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1779288644 0 0) 0x7fd8880053f0 con 0x7fd89c0744f0 2024-09-11T07:55:57.396 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.476+0000 7fd88dffb640 1 -- 172.21.15.102:0/1369898545 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd89c10f3b0 con 0x7fd89c0744f0 2024-09-11T07:55:57.396 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.476+0000 7fd88dffb640 1 -- 172.21.15.102:0/1369898545 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fd8980047a0 con 0x7fd89c0744f0 2024-09-11T07:55:57.396 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.477+0000 7fd88dffb640 1 -- 172.21.15.102:0/1369898545 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1370966609 0 0) 0x7fd89c10f3b0 con 0x7fd89c0744f0 2024-09-11T07:55:57.396 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.477+0000 7fd88dffb640 1 -- 172.21.15.102:0/1369898545 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c10f540 con 0x7fd89c0744f0 2024-09-11T07:55:57.396 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.477+0000 7fd8a37d7640 1 -- 172.21.15.102:0/1369898545 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd89c10f810 con 0x7fd89c0744f0 2024-09-11T07:55:57.396 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.477+0000 7fd88dffb640 1 -- 172.21.15.102:0/1369898545 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7fd898004ba0 con 0x7fd89c0744f0 2024-09-11T07:55:57.396 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.477+0000 7fd88dffb640 1 -- 172.21.15.102:0/1369898545 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fd898006180 con 0x7fd89c0744f0 2024-09-11T07:55:57.396 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.477+0000 7fd8a37d7640 1 -- 172.21.15.102:0/1369898545 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89c113250 con 0x7fd89c0744f0 2024-09-11T07:55:57.397 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.478+0000 7fd88dffb640 1 -- 172.21.15.102:0/1369898545 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 4) v1 ==== 45446+0+0 (unknown 4184903468 0 0) 0x7fd8980115c0 con 0x7fd89c0744f0 2024-09-11T07:55:57.397 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.478+0000 7fd88dffb640 1 -- 172.21.15.102:0/1369898545 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 1638648040 0 0) 0x7fd898047970 con 0x7fd89c0744f0 2024-09-11T07:55:57.397 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.478+0000 7fd8a37d7640 1 -- 172.21.15.102:0/1369898545 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd89c108a10 con 0x7fd89c0744f0 2024-09-11T07:55:57.397 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.483+0000 7fd88dffb640 1 -- 172.21.15.102:0/1369898545 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fd898013170 con 0x7fd89c0744f0 2024-09-11T07:55:57.397 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.677+0000 7fd8a37d7640 1 -- 172.21.15.102:0/1369898545 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fd89c105350 con 0x7fd89c0744f0 2024-09-11T07:55:57.397 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.730+0000 7fd88dffb640 1 -- 172.21.15.102:0/1369898545 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mgrmap(e 5) v1 ==== 45457+0+0 (unknown 3471134423 0 0) 0x7fd8980059a0 con 0x7fd89c0744f0 2024-09-11T07:55:57.397 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.731+0000 7fd88dffb640 1 -- 172.21.15.102:0/1369898545 <== mon.0 v1:172.21.15.102:6789/0 11 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (unknown 2263024820 0 0) 0x7fd898047350 con 0x7fd89c0744f0 2024-09-11T07:55:57.397 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.733+0000 7fd8a37d7640 1 -- 172.21.15.102:0/1369898545 >> v1:172.21.15.102:6800/3158655170 conn(0x7fd88803a650 legacy=0x7fd88803cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:55:57.397 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.733+0000 7fd8a37d7640 1 -- 172.21.15.102:0/1369898545 >> v1:172.21.15.102:6789/0 conn(0x7fd89c0744f0 legacy=0x7fd89c10eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:55:57.397 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.733+0000 7fd8a37d7640 1 -- 172.21.15.102:0/1369898545 shutdown_connections 2024-09-11T07:55:57.397 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.733+0000 7fd8a37d7640 1 -- 172.21.15.102:0/1369898545 >> 172.21.15.102:0/1369898545 conn(0x7fd89c0faa90 msgr2=0x7fd89c105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:55:57.397 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.734+0000 7fd8a37d7640 1 -- 172.21.15.102:0/1369898545 shutdown_connections 2024-09-11T07:55:57.397 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:56.734+0000 7fd8a37d7640 1 -- 172.21.15.102:0/1369898545 wait complete. 2024-09-11T07:55:57.755 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:57 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1369898545' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-11T07:55:57.755 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:57 smithi102 ceph-mon[29849]: mgrmap e5: y(active, since 6s) 2024-09-11T07:55:57.755 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:57 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:57.491+0000 7f027519bec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:55:57.756 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:57 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:57.552+0000 7f027519bec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:55:57.756 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:57 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:57.717+0000 7f027519bec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:55:58.626 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:58 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:58.570+0000 7f027519bec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:55:58.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:55:58.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-11T07:55:58.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T07:55:58.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-11T07:55:58.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T07:55:58.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:55:58.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.830+0000 7f57515a0640 1 Processor -- start 2024-09-11T07:55:58.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.830+0000 7f57515a0640 1 -- start start 2024-09-11T07:55:58.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.830+0000 7f57515a0640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f574c102350 con 0x7f574c101830 2024-09-11T07:55:58.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.830+0000 7f574bfff640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f574c101830 0x7f574c101c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57196/0 (socket says 172.21.15.102:57196) 2024-09-11T07:55:58.635 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.830+0000 7f574bfff640 1 -- 172.21.15.102:0/2431934236 learned_addr learned my addr 172.21.15.102:0/2431934236 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:55:58.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.830+0000 7f574affd640 1 -- 172.21.15.102:0/2431934236 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376314327 0 0) 0x7f574c102350 con 0x7f574c101830 2024-09-11T07:55:58.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.830+0000 7f574affd640 1 -- 172.21.15.102:0/2431934236 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57340034e0 con 0x7f574c101830 2024-09-11T07:55:58.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.830+0000 7f574affd640 1 -- 172.21.15.102:0/2431934236 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 467553150 0 0) 0x7f57340034e0 con 0x7f574c101830 2024-09-11T07:55:58.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.830+0000 7f574affd640 1 -- 172.21.15.102:0/2431934236 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c105840 con 0x7f574c101830 2024-09-11T07:55:58.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.830+0000 7f574affd640 1 -- 172.21.15.102:0/2431934236 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f573c001590 con 0x7f574c101830 2024-09-11T07:55:58.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.831+0000 7f574affd640 1 -- 172.21.15.102:0/2431934236 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7f573c001be0 con 0x7f574c101830 2024-09-11T07:55:58.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.831+0000 7f574affd640 1 -- 172.21.15.102:0/2431934236 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f573c0055b0 con 0x7f574c101830 2024-09-11T07:55:58.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.831+0000 7f57515a0640 1 -- 172.21.15.102:0/2431934236 >> v1:172.21.15.102:6789/0 conn(0x7f574c101830 legacy=0x7f574c101c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:55:58.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.831+0000 7f57515a0640 1 -- 172.21.15.102:0/2431934236 shutdown_connections 2024-09-11T07:55:58.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.831+0000 7f57515a0640 1 -- 172.21.15.102:0/2431934236 >> 172.21.15.102:0/2431934236 conn(0x7f574c0fa0d0 msgr2=0x7f574c0fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:55:58.636 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.831+0000 7f57515a0640 1 -- 172.21.15.102:0/2431934236 shutdown_connections 2024-09-11T07:55:58.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.831+0000 7f57515a0640 1 -- 172.21.15.102:0/2431934236 wait complete. 2024-09-11T07:55:58.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.832+0000 7f57515a0640 1 Processor -- start 2024-09-11T07:55:58.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.832+0000 7f57515a0640 1 -- start start 2024-09-11T07:55:58.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.832+0000 7f57515a0640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f574c10f240 con 0x7f574c101830 2024-09-11T07:55:58.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.832+0000 7f574bfff640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f574c101830 0x7f574c10eb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57206/0 (socket says 172.21.15.102:57206) 2024-09-11T07:55:58.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.832+0000 7f574bfff640 1 -- 172.21.15.102:0/1322489460 learned_addr learned my addr 172.21.15.102:0/1322489460 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:55:58.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.832+0000 7f5748ff9640 1 -- 172.21.15.102:0/1322489460 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2854758553 0 0) 0x7f574c10f240 con 0x7f574c101830 2024-09-11T07:55:58.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.832+0000 7f5748ff9640 1 -- 172.21.15.102:0/1322489460 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5734005440 con 0x7f574c101830 2024-09-11T07:55:58.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.833+0000 7f5748ff9640 1 -- 172.21.15.102:0/1322489460 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 954603817 0 0) 0x7f5734005440 con 0x7f574c101830 2024-09-11T07:55:58.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.833+0000 7f5748ff9640 1 -- 172.21.15.102:0/1322489460 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f574c10f240 con 0x7f574c101830 2024-09-11T07:55:58.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.833+0000 7f5748ff9640 1 -- 172.21.15.102:0/1322489460 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f573c0047a0 con 0x7f574c101830 2024-09-11T07:55:58.637 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.833+0000 7f5748ff9640 1 -- 172.21.15.102:0/1322489460 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 642739239 0 0) 0x7f574c10f240 con 0x7f574c101830 2024-09-11T07:55:58.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.833+0000 7f5748ff9640 1 -- 172.21.15.102:0/1322489460 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c10f3d0 con 0x7f574c101830 2024-09-11T07:55:58.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.833+0000 7f57515a0640 1 -- 172.21.15.102:0/1322489460 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574c10f6a0 con 0x7f574c101830 2024-09-11T07:55:58.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.833+0000 7f57515a0640 1 -- 172.21.15.102:0/1322489460 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f574c1132b0 con 0x7f574c101830 2024-09-11T07:55:58.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.833+0000 7f5748ff9640 1 -- 172.21.15.102:0/1322489460 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7f573c0017e0 con 0x7f574c101830 2024-09-11T07:55:58.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.833+0000 7f5748ff9640 1 -- 172.21.15.102:0/1322489460 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f573c006040 con 0x7f574c101830 2024-09-11T07:55:58.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.834+0000 7f5748ff9640 1 -- 172.21.15.102:0/1322489460 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 5) v1 ==== 45457+0+0 (unknown 3471134423 0 0) 0x7f573c011340 con 0x7f574c101830 2024-09-11T07:55:58.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.834+0000 7f57515a0640 1 -- 172.21.15.102:0/1322489460 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5738005040 con 0x7f574c101830 2024-09-11T07:55:58.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.834+0000 7f5748ff9640 1 -- 172.21.15.102:0/1322489460 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 1638648040 0 0) 0x7f573c047880 con 0x7f574c101830 2024-09-11T07:55:58.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.834+0000 7f574b7fe640 1 -- 172.21.15.102:0/1322489460 >> v1:172.21.15.102:6800/3158655170 conn(0x7f573403a790 legacy=0x7f573403cc40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.102:6800/3158655170 2024-09-11T07:55:58.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:57.839+0000 7f5748ff9640 1 -- 172.21.15.102:0/1322489460 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f573c016a90 con 0x7f574c101830 2024-09-11T07:55:58.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.035+0000 7f574b7fe640 1 -- 172.21.15.102:0/1322489460 >> v1:172.21.15.102:6800/3158655170 conn(0x7f573403a790 legacy=0x7f573403cc40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.102:6800/3158655170 2024-09-11T07:55:58.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.036+0000 7f57515a0640 1 -- 172.21.15.102:0/1322489460 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f5738005b00 con 0x7f574c101830 2024-09-11T07:55:58.638 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.037+0000 7f5748ff9640 1 -- 172.21.15.102:0/1322489460 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (unknown 3768197548 0 15966916) 0x7f573c016490 con 0x7f574c101830 2024-09-11T07:55:58.639 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.039+0000 7f57515a0640 1 -- 172.21.15.102:0/1322489460 >> v1:172.21.15.102:6800/3158655170 conn(0x7f573403a790 legacy=0x7f573403cc40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T07:55:58.639 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.039+0000 7f57515a0640 1 -- 172.21.15.102:0/1322489460 >> v1:172.21.15.102:6789/0 conn(0x7f574c101830 legacy=0x7f574c10eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:55:58.639 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.039+0000 7f57515a0640 1 -- 172.21.15.102:0/1322489460 shutdown_connections 2024-09-11T07:55:58.639 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.039+0000 7f57515a0640 1 -- 172.21.15.102:0/1322489460 >> 172.21.15.102:0/1322489460 conn(0x7f574c0fa0d0 msgr2=0x7f574c105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:55:58.639 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.039+0000 7f57515a0640 1 -- 172.21.15.102:0/1322489460 shutdown_connections 2024-09-11T07:55:58.639 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.039+0000 7f57515a0640 1 -- 172.21.15.102:0/1322489460 wait complete. 2024-09-11T07:55:58.639 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for the mgr to restart... 2024-09-11T07:55:58.639 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for mgr epoch 5... 2024-09-11T07:55:58.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:55:58 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1322489460' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T07:55:58.896 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:58 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:58.634+0000 7f027519bec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:55:59.455 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:59.345+0000 7f027519bec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:55:59.455 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:59.454+0000 7f027519bec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:55:59.753 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:59.572+0000 7f027519bec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:55:59.753 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:59.630+0000 7f027519bec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:55:59.754 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:59.752+0000 7f027519bec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:56:00.005 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:59.817+0000 7f027519bec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:56:00.005 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:55:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:55:59.873+0000 7f027519bec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:56:00.005 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:00 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:00.004+0000 7f027519bec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:56:00.264 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:00 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:00.066+0000 7f027519bec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:56:00.264 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:00 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:00.129+0000 7f027519bec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:56:00.265 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:00 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:00.207+0000 7f027519bec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:56:00.265 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:00 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:00.263+0000 7f027519bec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:56:00.645 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:00 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:00.319+0000 7f027519bec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:56:01.395 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:01 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:01.046+0000 7f027519bec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:56:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:01 smithi102 ceph-mon[29849]: Active manager daemon y restarted 2024-09-11T07:56:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:01 smithi102 ceph-mon[29849]: Activating manager daemon y 2024-09-11T07:56:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:01 smithi102 ceph-mon[29849]: osdmap e2: 0 total, 0 up, 0 in 2024-09-11T07:56:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:01 smithi102 ceph-mon[29849]: mgrmap e6: y(active, starting, since 0.00312188s) 2024-09-11T07:56:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:01 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:56:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:01 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:56:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:01 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:56:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:01 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:56:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:01 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:56:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:01 smithi102 ceph-mon[29849]: Manager daemon y is now available 2024-09-11T07:56:01.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:01 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' 2024-09-11T07:56:01.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:01 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' 2024-09-11T07:56:01.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:01 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:01.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:01 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:01.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:01 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:56:01.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:01 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:56:02.357 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:02 smithi102 ceph-mon[29849]: Found migration_current of "None". Setting to last migration. 2024-09-11T07:56:02.357 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:02 smithi102 ceph-mon[29849]: mgrmap e7: y(active, since 1.00516s) 2024-09-11T07:56:02.623 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:56:02.623 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-11T07:56:02.623 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T07:56:02.623 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:56:02.623 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.994+0000 7f93d1186640 1 Processor -- start 2024-09-11T07:56:02.623 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.994+0000 7f93d1186640 1 -- start start 2024-09-11T07:56:02.623 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.995+0000 7f93d1186640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f93cc0ff230 con 0x7f93cc0fe710 2024-09-11T07:56:02.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.995+0000 7f93cad9d640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f93cc0fe710 0x7f93cc0feb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57212/0 (socket says 172.21.15.102:57212) 2024-09-11T07:56:02.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.995+0000 7f93cad9d640 1 -- 172.21.15.102:0/3596883015 learned_addr learned my addr 172.21.15.102:0/3596883015 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:02.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.995+0000 7f93c9d9b640 1 -- 172.21.15.102:0/3596883015 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 368143745 0 0) 0x7f93cc0ff230 con 0x7f93cc0fe710 2024-09-11T07:56:02.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.995+0000 7f93c9d9b640 1 -- 172.21.15.102:0/3596883015 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93b40034e0 con 0x7f93cc0fe710 2024-09-11T07:56:02.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.995+0000 7f93c9d9b640 1 -- 172.21.15.102:0/3596883015 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2279669923 0 0) 0x7f93b40034e0 con 0x7f93cc0fe710 2024-09-11T07:56:02.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.995+0000 7f93c9d9b640 1 -- 172.21.15.102:0/3596883015 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93cc105a90 con 0x7f93cc0fe710 2024-09-11T07:56:02.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.995+0000 7f93c9d9b640 1 -- 172.21.15.102:0/3596883015 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f93bc001590 con 0x7f93cc0fe710 2024-09-11T07:56:02.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.995+0000 7f93c9d9b640 1 -- 172.21.15.102:0/3596883015 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7f93bc001cb0 con 0x7f93cc0fe710 2024-09-11T07:56:02.624 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.995+0000 7f93c9d9b640 1 -- 172.21.15.102:0/3596883015 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f93bc005670 con 0x7f93cc0fe710 2024-09-11T07:56:02.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.996+0000 7f93d1186640 1 -- 172.21.15.102:0/3596883015 >> v1:172.21.15.102:6789/0 conn(0x7f93cc0fe710 legacy=0x7f93cc0feb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:02.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.996+0000 7f93d1186640 1 -- 172.21.15.102:0/3596883015 shutdown_connections 2024-09-11T07:56:02.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.996+0000 7f93d1186640 1 -- 172.21.15.102:0/3596883015 >> 172.21.15.102:0/3596883015 conn(0x7f93cc0fa150 msgr2=0x7f93cc0fc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:02.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.996+0000 7f93d1186640 1 -- 172.21.15.102:0/3596883015 shutdown_connections 2024-09-11T07:56:02.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.996+0000 7f93d1186640 1 -- 172.21.15.102:0/3596883015 wait complete. 2024-09-11T07:56:02.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.997+0000 7f93d1186640 1 Processor -- start 2024-09-11T07:56:02.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.997+0000 7f93d1186640 1 -- start start 2024-09-11T07:56:02.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.997+0000 7f93d1186640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f93cc10f5c0 con 0x7f93cc0fe710 2024-09-11T07:56:02.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.997+0000 7f93cad9d640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f93cc0fe710 0x7f93cc10eeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57218/0 (socket says 172.21.15.102:57218) 2024-09-11T07:56:02.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.997+0000 7f93cad9d640 1 -- 172.21.15.102:0/2486099473 learned_addr learned my addr 172.21.15.102:0/2486099473 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:02.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.997+0000 7f93b37fe640 1 -- 172.21.15.102:0/2486099473 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1186927558 0 0) 0x7f93cc10f5c0 con 0x7f93cc0fe710 2024-09-11T07:56:02.625 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.997+0000 7f93b37fe640 1 -- 172.21.15.102:0/2486099473 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93b40053f0 con 0x7f93cc0fe710 2024-09-11T07:56:02.626 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.997+0000 7f93b37fe640 1 -- 172.21.15.102:0/2486099473 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2870245732 0 0) 0x7f93b40053f0 con 0x7f93cc0fe710 2024-09-11T07:56:02.626 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.997+0000 7f93b37fe640 1 -- 172.21.15.102:0/2486099473 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f93cc10f5c0 con 0x7f93cc0fe710 2024-09-11T07:56:02.626 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.997+0000 7f93b37fe640 1 -- 172.21.15.102:0/2486099473 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f93bc0047a0 con 0x7f93cc0fe710 2024-09-11T07:56:02.626 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.997+0000 7f93b37fe640 1 -- 172.21.15.102:0/2486099473 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2290500457 0 0) 0x7f93cc10f5c0 con 0x7f93cc0fe710 2024-09-11T07:56:02.626 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.998+0000 7f93b37fe640 1 -- 172.21.15.102:0/2486099473 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93cc10f750 con 0x7f93cc0fe710 2024-09-11T07:56:02.626 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.998+0000 7f93d1186640 1 -- 172.21.15.102:0/2486099473 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93cc10fa20 con 0x7f93cc0fe710 2024-09-11T07:56:02.626 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.998+0000 7f93d1186640 1 -- 172.21.15.102:0/2486099473 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f93cc1134b0 con 0x7f93cc0fe710 2024-09-11T07:56:02.626 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.998+0000 7f93b37fe640 1 -- 172.21.15.102:0/2486099473 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7f93bc0049e0 con 0x7f93cc0fe710 2024-09-11T07:56:02.626 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.998+0000 7f93b37fe640 1 -- 172.21.15.102:0/2486099473 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f93bc006060 con 0x7f93cc0fe710 2024-09-11T07:56:02.626 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.998+0000 7f93b37fe640 1 -- 172.21.15.102:0/2486099473 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 5) v1 ==== 45457+0+0 (unknown 3471134423 0 0) 0x7f93bc011360 con 0x7f93cc0fe710 2024-09-11T07:56:02.626 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.999+0000 7f93b37fe640 1 -- 172.21.15.102:0/2486099473 --> v1:172.21.15.102:6800/3158655170 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f93b403e450 con 0x7f93b403a790 2024-09-11T07:56:02.626 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.999+0000 7f93ca59c640 1 -- 172.21.15.102:0/2486099473 >> v1:172.21.15.102:6800/3158655170 conn(0x7f93b403a790 legacy=0x7f93b403cc40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.102:6800/3158655170 2024-09-11T07:56:02.626 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:58.999+0000 7f93b37fe640 1 -- 172.21.15.102:0/2486099473 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 1638648040 0 0) 0x7f93bc047820 con 0x7f93cc0fe710 2024-09-11T07:56:02.627 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:59.200+0000 7f93ca59c640 1 -- 172.21.15.102:0/2486099473 >> v1:172.21.15.102:6800/3158655170 conn(0x7f93b403a790 legacy=0x7f93b403cc40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.102:6800/3158655170 2024-09-11T07:56:02.627 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:55:59.601+0000 7f93ca59c640 1 -- 172.21.15.102:0/2486099473 >> v1:172.21.15.102:6800/3158655170 conn(0x7f93b403a790 legacy=0x7f93b403cc40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.102:6800/3158655170 2024-09-11T07:56:02.627 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:00.402+0000 7f93ca59c640 1 -- 172.21.15.102:0/2486099473 >> v1:172.21.15.102:6800/3158655170 conn(0x7f93b403a790 legacy=0x7f93b403cc40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.102:6800/3158655170 2024-09-11T07:56:02.627 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:01.050+0000 7f93b37fe640 1 -- 172.21.15.102:0/2486099473 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mgrmap(e 6) v1 ==== 45259+0+0 (unknown 2242559776 0 0) 0x7f93bc046910 con 0x7f93cc0fe710 2024-09-11T07:56:02.627 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:01.050+0000 7f93b37fe640 1 -- 172.21.15.102:0/2486099473 >> v1:172.21.15.102:6800/3158655170 conn(0x7f93b403a790 legacy=0x7f93b403cc40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T07:56:02.627 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:02.051+0000 7f93b37fe640 1 -- 172.21.15.102:0/2486099473 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mgrmap(e 7) v1 ==== 45351+0+0 (unknown 4074573174 0 0) 0x7f93bc0127c0 con 0x7f93cc0fe710 2024-09-11T07:56:02.627 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:02.051+0000 7f93b37fe640 1 -- 172.21.15.102:0/2486099473 --> v1:172.21.15.102:6800/3294079105 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f93b403e450 con 0x7f93b403efc0 2024-09-11T07:56:02.627 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:02.052+0000 7f93b37fe640 1 -- 172.21.15.102:0/2486099473 <== mgr.14118 v1:172.21.15.102:6800/3294079105 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (unknown 0 0 4193150809) 0x7f93b403e450 con 0x7f93b403efc0 2024-09-11T07:56:02.627 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:02.057+0000 7f93d1186640 1 -- 172.21.15.102:0/2486099473 --> v1:172.21.15.102:6800/3294079105 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f93b8002560 con 0x7f93b403efc0 2024-09-11T07:56:02.627 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:02.057+0000 7f93b37fe640 1 -- 172.21.15.102:0/2486099473 <== mgr.14118 v1:172.21.15.102:6800/3294079105 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (unknown 0 0 96372106) 0x7f93b8002560 con 0x7f93b403efc0 2024-09-11T07:56:02.627 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:02.058+0000 7f93d1186640 1 -- 172.21.15.102:0/2486099473 >> v1:172.21.15.102:6800/3294079105 conn(0x7f93b403efc0 legacy=0x7f93b40413a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:02.627 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:02.058+0000 7f93d1186640 1 -- 172.21.15.102:0/2486099473 >> v1:172.21.15.102:6789/0 conn(0x7f93cc0fe710 legacy=0x7f93cc10eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:02.627 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:02.058+0000 7f93d1186640 1 -- 172.21.15.102:0/2486099473 shutdown_connections 2024-09-11T07:56:02.628 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:02.058+0000 7f93d1186640 1 -- 172.21.15.102:0/2486099473 >> 172.21.15.102:0/2486099473 conn(0x7f93cc0fa150 msgr2=0x7f93cc105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:02.628 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:02.058+0000 7f93d1186640 1 -- 172.21.15.102:0/2486099473 shutdown_connections 2024-09-11T07:56:02.628 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:02.058+0000 7f93d1186640 1 -- 172.21.15.102:0/2486099473 wait complete. 2024-09-11T07:56:02.628 INFO:teuthology.orchestra.run.smithi102.stdout:mgr epoch 5 is available 2024-09-11T07:56:02.628 INFO:teuthology.orchestra.run.smithi102.stdout:Setting orchestrator backend to cephadm... 2024-09-11T07:56:02.877 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:02 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:02.678+0000 7f0227589640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:07:56:02] ENGINE Error in HTTPServer.serve 2024-09-11T07:56:02.878 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:02 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: Traceback (most recent call last): 2024-09-11T07:56:02.878 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:02 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:56:02.878 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:02 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: self._connections.run(self.expiration_interval) 2024-09-11T07:56:02.878 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:02 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:56:02.878 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:02 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: self._run(expiration_interval) 2024-09-11T07:56:02.878 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:02 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:56:02.878 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:02 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:56:02.878 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:02 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:56:02.878 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:02 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:56:02.878 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:02 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:56:02.878 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:02 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: s = self.context.wrap_socket( 2024-09-11T07:56:02.879 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:02 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:56:02.879 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:02 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: return self.sslsocket_class._create( 2024-09-11T07:56:02.879 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:02 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:56:02.879 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:02 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: self.do_handshake() 2024-09-11T07:56:02.879 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:02 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:56:02.879 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:02 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: self._sslobj.do_handshake() 2024-09-11T07:56:02.879 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:02 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:56:02.879 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:02 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:03.769 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.077+0000 7f036f1e5640 1 Processor -- start 2024-09-11T07:56:03.769 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.077+0000 7f036f1e5640 1 -- start start 2024-09-11T07:56:03.769 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.077+0000 7f036f1e5640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f03681046e0 con 0x7f03680744d0 2024-09-11T07:56:03.769 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.077+0000 7f036cf81640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f03680744d0 0x7f0368102fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57270/0 (socket says 172.21.15.102:57270) 2024-09-11T07:56:03.769 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.077+0000 7f036cf81640 1 -- 172.21.15.102:0/1044502305 learned_addr learned my addr 172.21.15.102:0/1044502305 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:03.770 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.077+0000 7f03677fe640 1 -- 172.21.15.102:0/1044502305 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 444254746 0 0) 0x7f03681046e0 con 0x7f03680744d0 2024-09-11T07:56:03.770 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.077+0000 7f03677fe640 1 -- 172.21.15.102:0/1044502305 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03580034e0 con 0x7f03680744d0 2024-09-11T07:56:03.770 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.078+0000 7f03677fe640 1 -- 172.21.15.102:0/1044502305 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1390392635 0 0) 0x7f03580034e0 con 0x7f03680744d0 2024-09-11T07:56:03.770 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.078+0000 7f03677fe640 1 -- 172.21.15.102:0/1044502305 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0368105880 con 0x7f03680744d0 2024-09-11T07:56:03.770 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.078+0000 7f03677fe640 1 -- 172.21.15.102:0/1044502305 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f035c001590 con 0x7f03680744d0 2024-09-11T07:56:03.770 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.078+0000 7f03677fe640 1 -- 172.21.15.102:0/1044502305 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7f035c001cb0 con 0x7f03680744d0 2024-09-11T07:56:03.770 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.078+0000 7f03677fe640 1 -- 172.21.15.102:0/1044502305 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f035c005600 con 0x7f03680744d0 2024-09-11T07:56:03.770 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.078+0000 7f036f1e5640 1 -- 172.21.15.102:0/1044502305 >> v1:172.21.15.102:6789/0 conn(0x7f03680744d0 legacy=0x7f0368102fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:03.770 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.078+0000 7f036f1e5640 1 -- 172.21.15.102:0/1044502305 shutdown_connections 2024-09-11T07:56:03.770 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.078+0000 7f036f1e5640 1 -- 172.21.15.102:0/1044502305 >> 172.21.15.102:0/1044502305 conn(0x7f03680faa90 msgr2=0x7f03680fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:03.770 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.078+0000 7f036f1e5640 1 -- 172.21.15.102:0/1044502305 shutdown_connections 2024-09-11T07:56:03.770 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.078+0000 7f036f1e5640 1 -- 172.21.15.102:0/1044502305 wait complete. 2024-09-11T07:56:03.770 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.079+0000 7f036f1e5640 1 Processor -- start 2024-09-11T07:56:03.771 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.079+0000 7f036f1e5640 1 -- start start 2024-09-11T07:56:03.771 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.079+0000 7f036f1e5640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f036810f280 con 0x7f03680744d0 2024-09-11T07:56:03.771 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.079+0000 7f036cf81640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f03680744d0 0x7f036810eb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57276/0 (socket says 172.21.15.102:57276) 2024-09-11T07:56:03.771 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.079+0000 7f036cf81640 1 -- 172.21.15.102:0/2511155569 learned_addr learned my addr 172.21.15.102:0/2511155569 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:03.771 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.080+0000 7f03657fa640 1 -- 172.21.15.102:0/2511155569 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 290876997 0 0) 0x7f036810f280 con 0x7f03680744d0 2024-09-11T07:56:03.771 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.080+0000 7f03657fa640 1 -- 172.21.15.102:0/2511155569 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03580053f0 con 0x7f03680744d0 2024-09-11T07:56:03.771 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.080+0000 7f03657fa640 1 -- 172.21.15.102:0/2511155569 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3205385788 0 0) 0x7f03580053f0 con 0x7f03680744d0 2024-09-11T07:56:03.771 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.080+0000 7f03657fa640 1 -- 172.21.15.102:0/2511155569 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f036810f280 con 0x7f03680744d0 2024-09-11T07:56:03.771 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.080+0000 7f03657fa640 1 -- 172.21.15.102:0/2511155569 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f035c0047a0 con 0x7f03680744d0 2024-09-11T07:56:03.771 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.080+0000 7f03657fa640 1 -- 172.21.15.102:0/2511155569 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 100857368 0 0) 0x7f036810f280 con 0x7f03680744d0 2024-09-11T07:56:03.771 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.080+0000 7f03657fa640 1 -- 172.21.15.102:0/2511155569 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036810f410 con 0x7f03680744d0 2024-09-11T07:56:03.771 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.080+0000 7f036f1e5640 1 -- 172.21.15.102:0/2511155569 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036810f6e0 con 0x7f03680744d0 2024-09-11T07:56:03.771 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.080+0000 7f036f1e5640 1 -- 172.21.15.102:0/2511155569 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f03681131c0 con 0x7f03680744d0 2024-09-11T07:56:03.772 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.080+0000 7f03657fa640 1 -- 172.21.15.102:0/2511155569 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7f035c004b30 con 0x7f03680744d0 2024-09-11T07:56:03.772 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.080+0000 7f03657fa640 1 -- 172.21.15.102:0/2511155569 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f035c006060 con 0x7f03680744d0 2024-09-11T07:56:03.772 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.081+0000 7f03657fa640 1 -- 172.21.15.102:0/2511155569 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 7) v1 ==== 45351+0+0 (unknown 4074573174 0 0) 0x7f035c0113d0 con 0x7f03680744d0 2024-09-11T07:56:03.772 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.081+0000 7f036f1e5640 1 -- 172.21.15.102:0/2511155569 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034c005040 con 0x7f03680744d0 2024-09-11T07:56:03.772 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.081+0000 7f03657fa640 1 -- 172.21.15.102:0/2511155569 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 1922949975 0 0) 0x7f035c0478c0 con 0x7f03680744d0 2024-09-11T07:56:03.772 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.086+0000 7f03657fa640 1 -- 172.21.15.102:0/2511155569 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f035c012c10 con 0x7f03680744d0 2024-09-11T07:56:03.772 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.254+0000 7f036f1e5640 1 -- 172.21.15.102:0/2511155569 --> v1:172.21.15.102:6800/3294079105 -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f034c000bc0 con 0x7f035803a530 2024-09-11T07:56:03.772 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.261+0000 7f03657fa640 1 -- 172.21.15.102:0/2511155569 <== mgr.14118 v1:172.21.15.102:6800/3294079105 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (unknown 0 0 0) 0x7f034c000bc0 con 0x7f035803a530 2024-09-11T07:56:03.772 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.263+0000 7f036f1e5640 1 -- 172.21.15.102:0/2511155569 >> v1:172.21.15.102:6800/3294079105 conn(0x7f035803a530 legacy=0x7f035803c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:03.772 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.263+0000 7f036f1e5640 1 -- 172.21.15.102:0/2511155569 >> v1:172.21.15.102:6789/0 conn(0x7f03680744d0 legacy=0x7f036810eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:03.772 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.263+0000 7f036f1e5640 1 -- 172.21.15.102:0/2511155569 shutdown_connections 2024-09-11T07:56:03.772 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.263+0000 7f036f1e5640 1 -- 172.21.15.102:0/2511155569 >> 172.21.15.102:0/2511155569 conn(0x7f03680faa90 msgr2=0x7f0368105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:03.772 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.263+0000 7f036f1e5640 1 -- 172.21.15.102:0/2511155569 shutdown_connections 2024-09-11T07:56:03.773 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:03.263+0000 7f036f1e5640 1 -- 172.21.15.102:0/2511155569 wait complete. 2024-09-11T07:56:03.970 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:03 smithi102 ceph-mon[29849]: from='client.14122 v1:172.21.15.102:0/2486099473' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T07:56:03.970 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:03 smithi102 ceph-mon[29849]: from='client.14122 v1:172.21.15.102:0/2486099473' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T07:56:03.970 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:03 smithi102 ceph-mon[29849]: [11/Sep/2024:07:56:02] ENGINE Bus STARTING 2024-09-11T07:56:03.970 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:03 smithi102 ceph-mon[29849]: [11/Sep/2024:07:56:02] ENGINE Serving on https://172.21.15.102:7150 2024-09-11T07:56:03.971 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:03 smithi102 ceph-mon[29849]: [11/Sep/2024:07:56:02] ENGINE Bus STARTED 2024-09-11T07:56:03.971 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:03 smithi102 ceph-mon[29849]: [11/Sep/2024:07:56:02] ENGINE Error in HTTPServer.serve 2024-09-11T07:56:03.971 INFO:journalctl@ceph.mon.a.smithi102.stdout: Traceback (most recent call last): 2024-09-11T07:56:03.971 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:56:03.971 INFO:journalctl@ceph.mon.a.smithi102.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:56:03.971 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:56:03.971 INFO:journalctl@ceph.mon.a.smithi102.stdout: self._run(expiration_interval) 2024-09-11T07:56:03.971 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:56:03.971 INFO:journalctl@ceph.mon.a.smithi102.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:56:03.971 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:56:03.971 INFO:journalctl@ceph.mon.a.smithi102.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:56:03.971 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:56:03.971 INFO:journalctl@ceph.mon.a.smithi102.stdout: s = self.context.wrap_socket( 2024-09-11T07:56:03.972 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:56:03.972 INFO:journalctl@ceph.mon.a.smithi102.stdout: return self.sslsocket_class._create( 2024-09-11T07:56:03.972 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:56:03.972 INFO:journalctl@ceph.mon.a.smithi102.stdout: self.do_handshake() 2024-09-11T07:56:03.972 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:56:03.972 INFO:journalctl@ceph.mon.a.smithi102.stdout: self._sslobj.do_handshake() 2024-09-11T07:56:03.972 INFO:journalctl@ceph.mon.a.smithi102.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:56:03.972 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:03 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' 2024-09-11T07:56:03.972 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:03 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:03.972 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:03 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' 2024-09-11T07:56:03.972 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:03 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:04.894 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:04 smithi102 ceph-mon[29849]: from='client.14130 v1:172.21.15.102:0/2511155569' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:56:04.894 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:04 smithi102 ceph-mon[29849]: mgrmap e8: y(active, since 2s) 2024-09-11T07:56:04.901 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-11T07:56:04.901 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.181+0000 7f64ef652640 1 Processor -- start 2024-09-11T07:56:04.901 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.182+0000 7f64ef652640 1 -- start start 2024-09-11T07:56:04.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.182+0000 7f64ef652640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f64e8100e90 con 0x7f64e8100370 2024-09-11T07:56:04.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.182+0000 7f64ed3ee640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f64e8100370 0x7f64e8100780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57288/0 (socket says 172.21.15.102:57288) 2024-09-11T07:56:04.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.182+0000 7f64ed3ee640 1 -- 172.21.15.102:0/687309882 learned_addr learned my addr 172.21.15.102:0/687309882 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:04.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.182+0000 7f64dffff640 1 -- 172.21.15.102:0/687309882 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903728501 0 0) 0x7f64e8100e90 con 0x7f64e8100370 2024-09-11T07:56:04.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.182+0000 7f64dffff640 1 -- 172.21.15.102:0/687309882 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64d40034e0 con 0x7f64e8100370 2024-09-11T07:56:04.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.183+0000 7f64dffff640 1 -- 172.21.15.102:0/687309882 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3166256919 0 0) 0x7f64d40034e0 con 0x7f64e8100370 2024-09-11T07:56:04.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.183+0000 7f64dffff640 1 -- 172.21.15.102:0/687309882 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e8105830 con 0x7f64e8100370 2024-09-11T07:56:04.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.183+0000 7f64dffff640 1 -- 172.21.15.102:0/687309882 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f64e4001590 con 0x7f64e8100370 2024-09-11T07:56:04.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.183+0000 7f64dffff640 1 -- 172.21.15.102:0/687309882 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7f64e4001cb0 con 0x7f64e8100370 2024-09-11T07:56:04.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.183+0000 7f64dffff640 1 -- 172.21.15.102:0/687309882 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f64e4005670 con 0x7f64e8100370 2024-09-11T07:56:04.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.183+0000 7f64ef652640 1 -- 172.21.15.102:0/687309882 >> v1:172.21.15.102:6789/0 conn(0x7f64e8100370 legacy=0x7f64e8100780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:04.902 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.183+0000 7f64ef652640 1 -- 172.21.15.102:0/687309882 shutdown_connections 2024-09-11T07:56:04.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.183+0000 7f64ef652640 1 -- 172.21.15.102:0/687309882 >> 172.21.15.102:0/687309882 conn(0x7f64e80fbbd0 msgr2=0x7f64e80fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:04.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.183+0000 7f64ef652640 1 -- 172.21.15.102:0/687309882 shutdown_connections 2024-09-11T07:56:04.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.184+0000 7f64ef652640 1 -- 172.21.15.102:0/687309882 wait complete. 2024-09-11T07:56:04.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.184+0000 7f64ef652640 1 Processor -- start 2024-09-11T07:56:04.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.184+0000 7f64ef652640 1 -- start start 2024-09-11T07:56:04.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.184+0000 7f64ef652640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f64e810f350 con 0x7f64e8100370 2024-09-11T07:56:04.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.184+0000 7f64ed3ee640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f64e8100370 0x7f64e810ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57300/0 (socket says 172.21.15.102:57300) 2024-09-11T07:56:04.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.184+0000 7f64ed3ee640 1 -- 172.21.15.102:0/698425313 learned_addr learned my addr 172.21.15.102:0/698425313 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:04.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.184+0000 7f64ddffb640 1 -- 172.21.15.102:0/698425313 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 628997884 0 0) 0x7f64e810f350 con 0x7f64e8100370 2024-09-11T07:56:04.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.185+0000 7f64ddffb640 1 -- 172.21.15.102:0/698425313 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64d40053f0 con 0x7f64e8100370 2024-09-11T07:56:04.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.185+0000 7f64ddffb640 1 -- 172.21.15.102:0/698425313 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3214750080 0 0) 0x7f64d40053f0 con 0x7f64e8100370 2024-09-11T07:56:04.903 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.185+0000 7f64ddffb640 1 -- 172.21.15.102:0/698425313 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f64e810f350 con 0x7f64e8100370 2024-09-11T07:56:04.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.185+0000 7f64ddffb640 1 -- 172.21.15.102:0/698425313 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f64e4001060 con 0x7f64e8100370 2024-09-11T07:56:04.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.185+0000 7f64ddffb640 1 -- 172.21.15.102:0/698425313 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2598853476 0 0) 0x7f64e810f350 con 0x7f64e8100370 2024-09-11T07:56:04.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.185+0000 7f64ddffb640 1 -- 172.21.15.102:0/698425313 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e810f4e0 con 0x7f64e8100370 2024-09-11T07:56:04.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.185+0000 7f64ef652640 1 -- 172.21.15.102:0/698425313 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e810f7b0 con 0x7f64e8100370 2024-09-11T07:56:04.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.185+0000 7f64ddffb640 1 -- 172.21.15.102:0/698425313 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7f64e4004ba0 con 0x7f64e8100370 2024-09-11T07:56:04.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.185+0000 7f64ddffb640 1 -- 172.21.15.102:0/698425313 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f64e40061a0 con 0x7f64e8100370 2024-09-11T07:56:04.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.185+0000 7f64ef652640 1 -- 172.21.15.102:0/698425313 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e8113290 con 0x7f64e8100370 2024-09-11T07:56:04.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.186+0000 7f64ddffb640 1 -- 172.21.15.102:0/698425313 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 8) v1 ==== 45457+0+0 (unknown 296826312 0 0) 0x7f64e40115f0 con 0x7f64e8100370 2024-09-11T07:56:04.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.186+0000 7f64ddffb640 1 -- 172.21.15.102:0/698425313 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 1922949975 0 0) 0x7f64e40487e0 con 0x7f64e8100370 2024-09-11T07:56:04.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.186+0000 7f64ef652640 1 -- 172.21.15.102:0/698425313 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e805ed70 con 0x7f64e8100370 2024-09-11T07:56:04.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.191+0000 7f64ddffb640 1 -- 172.21.15.102:0/698425313 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f64e40131f0 con 0x7f64e8100370 2024-09-11T07:56:04.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.345+0000 7f64ef652640 1 -- 172.21.15.102:0/698425313 --> v1:172.21.15.102:6800/3294079105 -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f64e802ab50 con 0x7f64d403a650 2024-09-11T07:56:04.904 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.345+0000 7f64ddffb640 1 -- 172.21.15.102:0/698425313 <== mgr.14118 v1:172.21.15.102:6800/3294079105 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (unknown 0 0 2070689548) 0x7f64e802ab50 con 0x7f64d403a650 2024-09-11T07:56:04.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.348+0000 7f64ef652640 1 -- 172.21.15.102:0/698425313 >> v1:172.21.15.102:6800/3294079105 conn(0x7f64d403a650 legacy=0x7f64d403cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:04.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.348+0000 7f64ef652640 1 -- 172.21.15.102:0/698425313 >> v1:172.21.15.102:6789/0 conn(0x7f64e8100370 legacy=0x7f64e810ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:04.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.348+0000 7f64ef652640 1 -- 172.21.15.102:0/698425313 shutdown_connections 2024-09-11T07:56:04.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.348+0000 7f64ef652640 1 -- 172.21.15.102:0/698425313 >> 172.21.15.102:0/698425313 conn(0x7f64e80fbbd0 msgr2=0x7f64e8105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:04.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.348+0000 7f64ef652640 1 -- 172.21.15.102:0/698425313 shutdown_connections 2024-09-11T07:56:04.905 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:04.348+0000 7f64ef652640 1 -- 172.21.15.102:0/698425313 wait complete. 2024-09-11T07:56:04.905 INFO:teuthology.orchestra.run.smithi102.stdout:Generating ssh key... 2024-09-11T07:56:06.067 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:05 smithi102 ceph-mon[29849]: from='client.14132 v1:172.21.15.102:0/698425313' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:56:06.067 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:05 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: Generating public/private rsa key pair. 2024-09-11T07:56:06.067 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:05 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: Your identification has been saved in /tmp/tmp7m56ncfs/key 2024-09-11T07:56:06.067 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:05 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: Your public key has been saved in /tmp/tmp7m56ncfs/key.pub 2024-09-11T07:56:06.067 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:05 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: The key fingerprint is: 2024-09-11T07:56:06.067 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:05 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: SHA256:0M5MIaRnXjiPT1A9jia+zD8nMWqKDcK71QFij3cPQ+c ceph-2051892c-7013-11ef-bcea-c7b262605968 2024-09-11T07:56:06.067 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:05 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: The key's randomart image is: 2024-09-11T07:56:06.067 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:05 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: +---[RSA 3072]----+ 2024-09-11T07:56:06.067 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:05 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: | .o o. | 2024-09-11T07:56:06.068 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:05 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: | . = .o | 2024-09-11T07:56:06.068 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:05 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: | o ...B.+o . | 2024-09-11T07:56:06.068 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:05 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: |. + o++%o . | 2024-09-11T07:56:06.068 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:05 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: | . o *oES | 2024-09-11T07:56:06.068 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:05 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: |. . o *oo | 2024-09-11T07:56:06.068 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:05 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: |.... + +.o | 2024-09-11T07:56:06.068 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:05 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: | .o+ * o . | 2024-09-11T07:56:06.068 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:05 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: | oo oo ..+ | 2024-09-11T07:56:06.068 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:05 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: +----[SHA256]-----+ 2024-09-11T07:56:06.335 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.312+0000 7f642a033640 1 Processor -- start 2024-09-11T07:56:06.335 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.312+0000 7f642a033640 1 -- start start 2024-09-11T07:56:06.335 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.312+0000 7f642a033640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f64241046c0 con 0x7f6424073c10 2024-09-11T07:56:06.335 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.312+0000 7f64237fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f6424073c10 0x7f6424102fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57310/0 (socket says 172.21.15.102:57310) 2024-09-11T07:56:06.335 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.312+0000 7f64237fe640 1 -- 172.21.15.102:0/150065848 learned_addr learned my addr 172.21.15.102:0/150065848 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:06.335 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.312+0000 7f64227fc640 1 -- 172.21.15.102:0/150065848 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 533459040 0 0) 0x7f64241046c0 con 0x7f6424073c10 2024-09-11T07:56:06.336 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.312+0000 7f64227fc640 1 -- 172.21.15.102:0/150065848 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64100034e0 con 0x7f6424073c10 2024-09-11T07:56:06.336 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.312+0000 7f64227fc640 1 -- 172.21.15.102:0/150065848 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4106063935 0 0) 0x7f64100034e0 con 0x7f6424073c10 2024-09-11T07:56:06.336 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.312+0000 7f64227fc640 1 -- 172.21.15.102:0/150065848 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6424105860 con 0x7f6424073c10 2024-09-11T07:56:06.336 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.312+0000 7f64227fc640 1 -- 172.21.15.102:0/150065848 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f6414001590 con 0x7f6424073c10 2024-09-11T07:56:06.336 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.313+0000 7f64227fc640 1 -- 172.21.15.102:0/150065848 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7f6414001cb0 con 0x7f6424073c10 2024-09-11T07:56:06.336 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.313+0000 7f64227fc640 1 -- 172.21.15.102:0/150065848 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f6414005670 con 0x7f6424073c10 2024-09-11T07:56:06.336 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.313+0000 7f642a033640 1 -- 172.21.15.102:0/150065848 >> v1:172.21.15.102:6789/0 conn(0x7f6424073c10 legacy=0x7f6424102fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:06.336 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.313+0000 7f642a033640 1 -- 172.21.15.102:0/150065848 shutdown_connections 2024-09-11T07:56:06.336 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.313+0000 7f642a033640 1 -- 172.21.15.102:0/150065848 >> 172.21.15.102:0/150065848 conn(0x7f64240fa9d0 msgr2=0x7f64240fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:06.337 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.313+0000 7f642a033640 1 -- 172.21.15.102:0/150065848 shutdown_connections 2024-09-11T07:56:06.337 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.313+0000 7f642a033640 1 -- 172.21.15.102:0/150065848 wait complete. 2024-09-11T07:56:06.337 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.314+0000 7f642a033640 1 Processor -- start 2024-09-11T07:56:06.337 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.314+0000 7f642a033640 1 -- start start 2024-09-11T07:56:06.337 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.314+0000 7f642a033640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6424100e00 con 0x7f6424073c10 2024-09-11T07:56:06.337 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.314+0000 7f64237fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f6424073c10 0x7f64241006f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57314/0 (socket says 172.21.15.102:57314) 2024-09-11T07:56:06.337 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.314+0000 7f64237fe640 1 -- 172.21.15.102:0/4168699828 learned_addr learned my addr 172.21.15.102:0/4168699828 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:06.338 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.314+0000 7f641bfff640 1 -- 172.21.15.102:0/4168699828 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 520472435 0 0) 0x7f6424100e00 con 0x7f6424073c10 2024-09-11T07:56:06.338 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.314+0000 7f641bfff640 1 -- 172.21.15.102:0/4168699828 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64100053f0 con 0x7f6424073c10 2024-09-11T07:56:06.338 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.314+0000 7f641bfff640 1 -- 172.21.15.102:0/4168699828 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4056382583 0 0) 0x7f64100053f0 con 0x7f6424073c10 2024-09-11T07:56:06.338 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.314+0000 7f641bfff640 1 -- 172.21.15.102:0/4168699828 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6424100e00 con 0x7f6424073c10 2024-09-11T07:56:06.338 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.314+0000 7f641bfff640 1 -- 172.21.15.102:0/4168699828 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f64140047a0 con 0x7f6424073c10 2024-09-11T07:56:06.338 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.315+0000 7f641bfff640 1 -- 172.21.15.102:0/4168699828 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1415118939 0 0) 0x7f6424100e00 con 0x7f6424073c10 2024-09-11T07:56:06.338 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.315+0000 7f641bfff640 1 -- 172.21.15.102:0/4168699828 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6424115d50 con 0x7f6424073c10 2024-09-11T07:56:06.338 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.315+0000 7f642a033640 1 -- 172.21.15.102:0/4168699828 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6424114da0 con 0x7f6424073c10 2024-09-11T07:56:06.338 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.315+0000 7f642a033640 1 -- 172.21.15.102:0/4168699828 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6424115270 con 0x7f6424073c10 2024-09-11T07:56:06.338 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.315+0000 7f641bfff640 1 -- 172.21.15.102:0/4168699828 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7f64140049c0 con 0x7f6424073c10 2024-09-11T07:56:06.338 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.315+0000 7f641bfff640 1 -- 172.21.15.102:0/4168699828 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f64140060d0 con 0x7f6424073c10 2024-09-11T07:56:06.338 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.316+0000 7f641bfff640 1 -- 172.21.15.102:0/4168699828 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 8) v1 ==== 45457+0+0 (unknown 296826312 0 0) 0x7f6414006380 con 0x7f6424073c10 2024-09-11T07:56:06.339 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.316+0000 7f642a033640 1 -- 172.21.15.102:0/4168699828 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6404005040 con 0x7f6424073c10 2024-09-11T07:56:06.339 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.316+0000 7f641bfff640 1 -- 172.21.15.102:0/4168699828 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 1922949975 0 0) 0x7f64140479b0 con 0x7f6424073c10 2024-09-11T07:56:06.339 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.320+0000 7f641bfff640 1 -- 172.21.15.102:0/4168699828 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f6414007170 con 0x7f6424073c10 2024-09-11T07:56:06.339 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.478+0000 7f642a033640 1 -- 172.21.15.102:0/4168699828 --> v1:172.21.15.102:6800/3294079105 -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f6404000bc0 con 0x7f641003a6f0 2024-09-11T07:56:06.339 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.754+0000 7f641bfff640 1 -- 172.21.15.102:0/4168699828 <== mgr.14118 v1:172.21.15.102:6800/3294079105 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (unknown 0 0 0) 0x7f6404000bc0 con 0x7f641003a6f0 2024-09-11T07:56:06.339 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.756+0000 7f642a033640 1 -- 172.21.15.102:0/4168699828 >> v1:172.21.15.102:6800/3294079105 conn(0x7f641003a6f0 legacy=0x7f641003cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:06.339 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.756+0000 7f642a033640 1 -- 172.21.15.102:0/4168699828 >> v1:172.21.15.102:6789/0 conn(0x7f6424073c10 legacy=0x7f64241006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:06.339 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.756+0000 7f642a033640 1 -- 172.21.15.102:0/4168699828 shutdown_connections 2024-09-11T07:56:06.339 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.756+0000 7f642a033640 1 -- 172.21.15.102:0/4168699828 >> 172.21.15.102:0/4168699828 conn(0x7f64240fa9d0 msgr2=0x7f64241028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:06.339 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.756+0000 7f642a033640 1 -- 172.21.15.102:0/4168699828 shutdown_connections 2024-09-11T07:56:06.339 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:05.756+0000 7f642a033640 1 -- 172.21.15.102:0/4168699828 wait complete. 2024-09-11T07:56:06.923 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:06 smithi102 ceph-mon[29849]: from='client.14134 v1:172.21.15.102:0/4168699828' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:56:06.923 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:06 smithi102 ceph-mon[29849]: Generating ssh key... 2024-09-11T07:56:06.923 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:06 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' 2024-09-11T07:56:06.923 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:06 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' 2024-09-11T07:56:07.515 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCJazO2dLR6bekFDB1HF6xtX0o8yKt/esGCHLtfZ2JxW0lik2kFcVuCzE+dyPkvmeEW1jzh8GPCYe9yVHaF3kFic48Y2xTcvapMaxulEpfrCXRUUWMP3kdFe6FUq2p5aOMga9CaCikRKyQlYJ5cc/yoXJBcWTfk0cB1JnL12KUJKaD9zpPZt89PMihSvA9lMIEAuafgnid8nvl5tQpLEMk8HyPW7xGsm9pyO7AMfBPFngaoGlmAaEa1o/iOYMOgcIs8RX2vjgmOuMFu/P1a7B/U9ZU1ZMV7NPGwtjmNuR85Q2C5kvXXQrGPFRqSUfi/dSTJ21kkvGiuRYMVlJ004dt0BamTxy9OTlw3pBt4CiDJNyqjR6wssONrXb43jyiuUPKKDFCAfS8MEbAFPLSHIUx0irgXTRd89J8ThFhl2URuEj4DoYxIE4tnvpfxBQ77qIfgU3MuapC71Cqyk52XMaTby9IpgtWMLEbDCEhSuAY76x71a43MJQrraIbcuX2tHLU= ceph-2051892c-7013-11ef-bcea-c7b262605968 2024-09-11T07:56:07.515 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.745+0000 7fda85613640 1 Processor -- start 2024-09-11T07:56:07.515 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.745+0000 7fda85613640 1 -- start start 2024-09-11T07:56:07.515 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.745+0000 7fda85613640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fda80076990 con 0x7fda80076580 2024-09-11T07:56:07.515 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.746+0000 7fda7effd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fda80076580 0x7fda80074b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57324/0 (socket says 172.21.15.102:57324) 2024-09-11T07:56:07.515 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.746+0000 7fda7effd640 1 -- 172.21.15.102:0/3747492972 learned_addr learned my addr 172.21.15.102:0/3747492972 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:07.515 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.746+0000 7fda7dffb640 1 -- 172.21.15.102:0/3747492972 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3066178508 0 0) 0x7fda80076990 con 0x7fda80076580 2024-09-11T07:56:07.515 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.746+0000 7fda7dffb640 1 -- 172.21.15.102:0/3747492972 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda6c0034e0 con 0x7fda80076580 2024-09-11T07:56:07.516 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.746+0000 7fda7dffb640 1 -- 172.21.15.102:0/3747492972 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4228580383 0 0) 0x7fda6c0034e0 con 0x7fda80076580 2024-09-11T07:56:07.516 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.746+0000 7fda7dffb640 1 -- 172.21.15.102:0/3747492972 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda80075210 con 0x7fda80076580 2024-09-11T07:56:07.516 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.746+0000 7fda7dffb640 1 -- 172.21.15.102:0/3747492972 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fda70001910 con 0x7fda80076580 2024-09-11T07:56:07.516 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.746+0000 7fda7dffb640 1 -- 172.21.15.102:0/3747492972 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7fda70001f60 con 0x7fda80076580 2024-09-11T07:56:07.516 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.746+0000 7fda7dffb640 1 -- 172.21.15.102:0/3747492972 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fda70005930 con 0x7fda80076580 2024-09-11T07:56:07.516 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.747+0000 7fda85613640 1 -- 172.21.15.102:0/3747492972 >> v1:172.21.15.102:6789/0 conn(0x7fda80076580 legacy=0x7fda80074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:07.516 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.747+0000 7fda85613640 1 -- 172.21.15.102:0/3747492972 shutdown_connections 2024-09-11T07:56:07.516 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.747+0000 7fda85613640 1 -- 172.21.15.102:0/3747492972 >> 172.21.15.102:0/3747492972 conn(0x7fda800fbb70 msgr2=0x7fda800fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:07.516 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.747+0000 7fda85613640 1 -- 172.21.15.102:0/3747492972 shutdown_connections 2024-09-11T07:56:07.516 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.747+0000 7fda85613640 1 -- 172.21.15.102:0/3747492972 wait complete. 2024-09-11T07:56:07.516 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.747+0000 7fda85613640 1 Processor -- start 2024-09-11T07:56:07.516 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.747+0000 7fda85613640 1 -- start start 2024-09-11T07:56:07.516 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.748+0000 7fda85613640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fda801115e0 con 0x7fda80076580 2024-09-11T07:56:07.517 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.748+0000 7fda7effd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fda80076580 0x7fda80110ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57328/0 (socket says 172.21.15.102:57328) 2024-09-11T07:56:07.517 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.748+0000 7fda7effd640 1 -- 172.21.15.102:0/1093630409 learned_addr learned my addr 172.21.15.102:0/1093630409 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:07.517 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.750+0000 7fda777fe640 1 -- 172.21.15.102:0/1093630409 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 977257286 0 0) 0x7fda801115e0 con 0x7fda80076580 2024-09-11T07:56:07.517 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.750+0000 7fda777fe640 1 -- 172.21.15.102:0/1093630409 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda6c005440 con 0x7fda80076580 2024-09-11T07:56:07.517 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.750+0000 7fda777fe640 1 -- 172.21.15.102:0/1093630409 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3299864970 0 0) 0x7fda6c005440 con 0x7fda80076580 2024-09-11T07:56:07.517 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.750+0000 7fda777fe640 1 -- 172.21.15.102:0/1093630409 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fda801115e0 con 0x7fda80076580 2024-09-11T07:56:07.517 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.750+0000 7fda777fe640 1 -- 172.21.15.102:0/1093630409 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fda700020d0 con 0x7fda80076580 2024-09-11T07:56:07.517 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.750+0000 7fda777fe640 1 -- 172.21.15.102:0/1093630409 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1657362603 0 0) 0x7fda801115e0 con 0x7fda80076580 2024-09-11T07:56:07.517 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.750+0000 7fda777fe640 1 -- 172.21.15.102:0/1093630409 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda80111770 con 0x7fda80076580 2024-09-11T07:56:07.517 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.750+0000 7fda85613640 1 -- 172.21.15.102:0/1093630409 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda80111a40 con 0x7fda80076580 2024-09-11T07:56:07.517 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.750+0000 7fda85613640 1 -- 172.21.15.102:0/1093630409 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fda80115520 con 0x7fda80076580 2024-09-11T07:56:07.517 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.750+0000 7fda777fe640 1 -- 172.21.15.102:0/1093630409 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7fda70001b60 con 0x7fda80076580 2024-09-11T07:56:07.517 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.750+0000 7fda777fe640 1 -- 172.21.15.102:0/1093630409 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fda70006390 con 0x7fda80076580 2024-09-11T07:56:07.518 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.751+0000 7fda777fe640 1 -- 172.21.15.102:0/1093630409 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 8) v1 ==== 45457+0+0 (unknown 296826312 0 0) 0x7fda70011750 con 0x7fda80076580 2024-09-11T07:56:07.518 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.752+0000 7fda85613640 1 -- 172.21.15.102:0/1093630409 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda60005040 con 0x7fda80076580 2024-09-11T07:56:07.518 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.752+0000 7fda777fe640 1 -- 172.21.15.102:0/1093630409 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 1922949975 0 0) 0x7fda70047e40 con 0x7fda80076580 2024-09-11T07:56:07.518 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.756+0000 7fda777fe640 1 -- 172.21.15.102:0/1093630409 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fda70012f90 con 0x7fda80076580 2024-09-11T07:56:07.518 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.910+0000 7fda85613640 1 -- 172.21.15.102:0/1093630409 --> v1:172.21.15.102:6800/3294079105 -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fda60000bc0 con 0x7fda6c03a650 2024-09-11T07:56:07.518 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.911+0000 7fda777fe640 1 -- 172.21.15.102:0/1093630409 <== mgr.14118 v1:172.21.15.102:6800/3294079105 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (unknown 0 0 978369748) 0x7fda60000bc0 con 0x7fda6c03a650 2024-09-11T07:56:07.518 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.912+0000 7fda85613640 1 -- 172.21.15.102:0/1093630409 >> v1:172.21.15.102:6800/3294079105 conn(0x7fda6c03a650 legacy=0x7fda6c03cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:07.518 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.912+0000 7fda85613640 1 -- 172.21.15.102:0/1093630409 >> v1:172.21.15.102:6789/0 conn(0x7fda80076580 legacy=0x7fda80110ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:07.518 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.912+0000 7fda85613640 1 -- 172.21.15.102:0/1093630409 shutdown_connections 2024-09-11T07:56:07.518 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.913+0000 7fda85613640 1 -- 172.21.15.102:0/1093630409 >> 172.21.15.102:0/1093630409 conn(0x7fda800fbb70 msgr2=0x7fda801053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:07.518 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.913+0000 7fda85613640 1 -- 172.21.15.102:0/1093630409 shutdown_connections 2024-09-11T07:56:07.518 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:06.913+0000 7fda85613640 1 -- 172.21.15.102:0/1093630409 wait complete. 2024-09-11T07:56:07.519 INFO:teuthology.orchestra.run.smithi102.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-11T07:56:07.519 INFO:teuthology.orchestra.run.smithi102.stdout:Adding key to root@localhost authorized_keys... 2024-09-11T07:56:07.521 INFO:teuthology.orchestra.run.smithi102.stdout:Adding host smithi102... 2024-09-11T07:56:08.117 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:07 smithi102 ceph-mon[29849]: from='client.14136 v1:172.21.15.102:0/1093630409' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:56:08.902 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:08 smithi102 ceph-mon[29849]: from='client.14138 v1:172.21.15.102:0/2683125974' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi102", "addr": "172.21.15.102", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:56:09.843 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:09 smithi102 ceph-mon[29849]: Deploying cephadm binary to smithi102 2024-09-11T07:56:11.348 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout Added host 'smithi102' with addr '172.21.15.102' 2024-09-11T07:56:11.348 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.947+0000 7fc613079640 1 Processor -- start 2024-09-11T07:56:11.348 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.947+0000 7fc613079640 1 -- start start 2024-09-11T07:56:11.348 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.947+0000 7fc613079640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc60c100e40 con 0x7fc60c100320 2024-09-11T07:56:11.348 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.947+0000 7fc610e15640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fc60c100320 0x7fc60c100730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46438/0 (socket says 172.21.15.102:46438) 2024-09-11T07:56:11.348 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.947+0000 7fc610e15640 1 -- 172.21.15.102:0/306975052 learned_addr learned my addr 172.21.15.102:0/306975052 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:11.349 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.947+0000 7fc60b7fe640 1 -- 172.21.15.102:0/306975052 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935105628 0 0) 0x7fc60c100e40 con 0x7fc60c100320 2024-09-11T07:56:11.349 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.947+0000 7fc60b7fe640 1 -- 172.21.15.102:0/306975052 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5fc0034e0 con 0x7fc60c100320 2024-09-11T07:56:11.349 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.948+0000 7fc60b7fe640 1 -- 172.21.15.102:0/306975052 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2789936220 0 0) 0x7fc5fc0034e0 con 0x7fc60c100320 2024-09-11T07:56:11.349 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.948+0000 7fc60b7fe640 1 -- 172.21.15.102:0/306975052 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60c105870 con 0x7fc60c100320 2024-09-11T07:56:11.349 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.948+0000 7fc60b7fe640 1 -- 172.21.15.102:0/306975052 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fc600001590 con 0x7fc60c100320 2024-09-11T07:56:11.349 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.948+0000 7fc60b7fe640 1 -- 172.21.15.102:0/306975052 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7fc600001cb0 con 0x7fc60c100320 2024-09-11T07:56:11.349 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.948+0000 7fc60b7fe640 1 -- 172.21.15.102:0/306975052 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fc600005670 con 0x7fc60c100320 2024-09-11T07:56:11.349 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.948+0000 7fc613079640 1 -- 172.21.15.102:0/306975052 >> v1:172.21.15.102:6789/0 conn(0x7fc60c100320 legacy=0x7fc60c100730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:11.349 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.948+0000 7fc613079640 1 -- 172.21.15.102:0/306975052 shutdown_connections 2024-09-11T07:56:11.349 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.948+0000 7fc613079640 1 -- 172.21.15.102:0/306975052 >> 172.21.15.102:0/306975052 conn(0x7fc60c0fbba0 msgr2=0x7fc60c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:11.349 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.948+0000 7fc613079640 1 -- 172.21.15.102:0/306975052 shutdown_connections 2024-09-11T07:56:11.349 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.948+0000 7fc613079640 1 -- 172.21.15.102:0/306975052 wait complete. 2024-09-11T07:56:11.350 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.949+0000 7fc613079640 1 Processor -- start 2024-09-11T07:56:11.350 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.949+0000 7fc613079640 1 -- start start 2024-09-11T07:56:11.350 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.949+0000 7fc613079640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc60c111530 con 0x7fc60c100320 2024-09-11T07:56:11.350 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.949+0000 7fc610e15640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fc60c100320 0x7fc60c110e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46440/0 (socket says 172.21.15.102:46440) 2024-09-11T07:56:11.350 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.949+0000 7fc610e15640 1 -- 172.21.15.102:0/2683125974 learned_addr learned my addr 172.21.15.102:0/2683125974 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:11.350 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.949+0000 7fc6097fa640 1 -- 172.21.15.102:0/2683125974 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3768889404 0 0) 0x7fc60c111530 con 0x7fc60c100320 2024-09-11T07:56:11.350 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.949+0000 7fc6097fa640 1 -- 172.21.15.102:0/2683125974 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5fc0053f0 con 0x7fc60c100320 2024-09-11T07:56:11.350 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.950+0000 7fc6097fa640 1 -- 172.21.15.102:0/2683125974 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2106662988 0 0) 0x7fc5fc0053f0 con 0x7fc60c100320 2024-09-11T07:56:11.350 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.950+0000 7fc6097fa640 1 -- 172.21.15.102:0/2683125974 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc60c111530 con 0x7fc60c100320 2024-09-11T07:56:11.350 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.950+0000 7fc6097fa640 1 -- 172.21.15.102:0/2683125974 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fc6000047a0 con 0x7fc60c100320 2024-09-11T07:56:11.350 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.950+0000 7fc6097fa640 1 -- 172.21.15.102:0/2683125974 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3510261117 0 0) 0x7fc60c111530 con 0x7fc60c100320 2024-09-11T07:56:11.350 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.950+0000 7fc6097fa640 1 -- 172.21.15.102:0/2683125974 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60c1116c0 con 0x7fc60c100320 2024-09-11T07:56:11.350 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.950+0000 7fc613079640 1 -- 172.21.15.102:0/2683125974 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc60c111990 con 0x7fc60c100320 2024-09-11T07:56:11.351 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.950+0000 7fc613079640 1 -- 172.21.15.102:0/2683125974 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc60c115470 con 0x7fc60c100320 2024-09-11T07:56:11.351 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.950+0000 7fc6097fa640 1 -- 172.21.15.102:0/2683125974 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7fc600004ba0 con 0x7fc60c100320 2024-09-11T07:56:11.351 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.950+0000 7fc6097fa640 1 -- 172.21.15.102:0/2683125974 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fc600006030 con 0x7fc60c100320 2024-09-11T07:56:11.351 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.951+0000 7fc6097fa640 1 -- 172.21.15.102:0/2683125974 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 8) v1 ==== 45457+0+0 (unknown 296826312 0 0) 0x7fc600011410 con 0x7fc60c100320 2024-09-11T07:56:11.351 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.951+0000 7fc613079640 1 -- 172.21.15.102:0/2683125974 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f0005040 con 0x7fc60c100320 2024-09-11T07:56:11.351 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.951+0000 7fc6097fa640 1 -- 172.21.15.102:0/2683125974 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 1922949975 0 0) 0x7fc600047a10 con 0x7fc60c100320 2024-09-11T07:56:11.351 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:07.955+0000 7fc6097fa640 1 -- 172.21.15.102:0/2683125974 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fc600012c50 con 0x7fc60c100320 2024-09-11T07:56:11.351 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:08.115+0000 7fc613079640 1 -- 172.21.15.102:0/2683125974 --> v1:172.21.15.102:6800/3294079105 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi102", "addr": "172.21.15.102", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5f0000bc0 con 0x7fc5fc03a650 2024-09-11T07:56:11.351 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:10.784+0000 7fc6097fa640 1 -- 172.21.15.102:0/2683125974 <== mgr.14118 v1:172.21.15.102:6800/3294079105 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (unknown 0 0 544975597) 0x7fc5f0000bc0 con 0x7fc5fc03a650 2024-09-11T07:56:11.351 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:10.786+0000 7fc613079640 1 -- 172.21.15.102:0/2683125974 >> v1:172.21.15.102:6800/3294079105 conn(0x7fc5fc03a650 legacy=0x7fc5fc03cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:11.351 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:10.786+0000 7fc613079640 1 -- 172.21.15.102:0/2683125974 >> v1:172.21.15.102:6789/0 conn(0x7fc60c100320 legacy=0x7fc60c110e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:11.351 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:10.786+0000 7fc613079640 1 -- 172.21.15.102:0/2683125974 shutdown_connections 2024-09-11T07:56:11.351 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:10.786+0000 7fc613079640 1 -- 172.21.15.102:0/2683125974 >> 172.21.15.102:0/2683125974 conn(0x7fc60c0fbba0 msgr2=0x7fc60c0fbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:11.352 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:10.786+0000 7fc613079640 1 -- 172.21.15.102:0/2683125974 shutdown_connections 2024-09-11T07:56:11.352 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:10.786+0000 7fc613079640 1 -- 172.21.15.102:0/2683125974 wait complete. 2024-09-11T07:56:11.352 INFO:teuthology.orchestra.run.smithi102.stdout:Deploying unmanaged mon service... 2024-09-11T07:56:11.914 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:11 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' 2024-09-11T07:56:11.914 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:11 smithi102 ceph-mon[29849]: Added host smithi102 2024-09-11T07:56:11.914 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:11 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:12.615 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-11T07:56:12.615 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.738+0000 7f2c7b24f640 1 Processor -- start 2024-09-11T07:56:12.615 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.738+0000 7f2c7b24f640 1 -- start start 2024-09-11T07:56:12.615 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.739+0000 7f2c7b24f640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2c74104700 con 0x7f2c74073bd0 2024-09-11T07:56:12.615 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.739+0000 7f2c78feb640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2c74073bd0 0x7f2c74102fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46454/0 (socket says 172.21.15.102:46454) 2024-09-11T07:56:12.615 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.739+0000 7f2c78feb640 1 -- 172.21.15.102:0/1019254290 learned_addr learned my addr 172.21.15.102:0/1019254290 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:12.615 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.739+0000 7f2c6b7fe640 1 -- 172.21.15.102:0/1019254290 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3081532416 0 0) 0x7f2c74104700 con 0x7f2c74073bd0 2024-09-11T07:56:12.615 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.739+0000 7f2c6b7fe640 1 -- 172.21.15.102:0/1019254290 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c5c0034e0 con 0x7f2c74073bd0 2024-09-11T07:56:12.616 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.739+0000 7f2c6b7fe640 1 -- 172.21.15.102:0/1019254290 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 696037739 0 0) 0x7f2c5c0034e0 con 0x7f2c74073bd0 2024-09-11T07:56:12.616 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.739+0000 7f2c6b7fe640 1 -- 172.21.15.102:0/1019254290 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c741058a0 con 0x7f2c74073bd0 2024-09-11T07:56:12.616 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.739+0000 7f2c6b7fe640 1 -- 172.21.15.102:0/1019254290 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f2c70001590 con 0x7f2c74073bd0 2024-09-11T07:56:12.616 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.739+0000 7f2c6b7fe640 1 -- 172.21.15.102:0/1019254290 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7f2c70001cb0 con 0x7f2c74073bd0 2024-09-11T07:56:12.616 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.739+0000 7f2c6b7fe640 1 -- 172.21.15.102:0/1019254290 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f2c70005670 con 0x7f2c74073bd0 2024-09-11T07:56:12.616 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.740+0000 7f2c7b24f640 1 -- 172.21.15.102:0/1019254290 >> v1:172.21.15.102:6789/0 conn(0x7f2c74073bd0 legacy=0x7f2c74102fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:12.616 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.740+0000 7f2c7b24f640 1 -- 172.21.15.102:0/1019254290 shutdown_connections 2024-09-11T07:56:12.616 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.740+0000 7f2c7b24f640 1 -- 172.21.15.102:0/1019254290 >> 172.21.15.102:0/1019254290 conn(0x7f2c740fa9d0 msgr2=0x7f2c740fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:12.616 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.740+0000 7f2c7b24f640 1 -- 172.21.15.102:0/1019254290 shutdown_connections 2024-09-11T07:56:12.616 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.740+0000 7f2c7b24f640 1 -- 172.21.15.102:0/1019254290 wait complete. 2024-09-11T07:56:12.616 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.740+0000 7f2c7b24f640 1 Processor -- start 2024-09-11T07:56:12.616 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.740+0000 7f2c7b24f640 1 -- start start 2024-09-11T07:56:12.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.741+0000 7f2c7b24f640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2c7410f390 con 0x7f2c74073bd0 2024-09-11T07:56:12.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.741+0000 7f2c78feb640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2c74073bd0 0x7f2c7410ec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46460/0 (socket says 172.21.15.102:46460) 2024-09-11T07:56:12.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.741+0000 7f2c78feb640 1 -- 172.21.15.102:0/3994405499 learned_addr learned my addr 172.21.15.102:0/3994405499 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:12.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.741+0000 7f2c697fa640 1 -- 172.21.15.102:0/3994405499 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1728033603 0 0) 0x7f2c7410f390 con 0x7f2c74073bd0 2024-09-11T07:56:12.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.741+0000 7f2c697fa640 1 -- 172.21.15.102:0/3994405499 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c5c0053f0 con 0x7f2c74073bd0 2024-09-11T07:56:12.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.741+0000 7f2c697fa640 1 -- 172.21.15.102:0/3994405499 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1566292208 0 0) 0x7f2c5c0053f0 con 0x7f2c74073bd0 2024-09-11T07:56:12.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.741+0000 7f2c697fa640 1 -- 172.21.15.102:0/3994405499 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2c7410f390 con 0x7f2c74073bd0 2024-09-11T07:56:12.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.741+0000 7f2c697fa640 1 -- 172.21.15.102:0/3994405499 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f2c700047a0 con 0x7f2c74073bd0 2024-09-11T07:56:12.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.741+0000 7f2c697fa640 1 -- 172.21.15.102:0/3994405499 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1089402946 0 0) 0x7f2c7410f390 con 0x7f2c74073bd0 2024-09-11T07:56:12.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.741+0000 7f2c697fa640 1 -- 172.21.15.102:0/3994405499 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7410f520 con 0x7f2c74073bd0 2024-09-11T07:56:12.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.741+0000 7f2c7b24f640 1 -- 172.21.15.102:0/3994405499 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7410f7f0 con 0x7f2c74073bd0 2024-09-11T07:56:12.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.741+0000 7f2c7b24f640 1 -- 172.21.15.102:0/3994405499 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c74113280 con 0x7f2c74073bd0 2024-09-11T07:56:12.617 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.741+0000 7f2c697fa640 1 -- 172.21.15.102:0/3994405499 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7f2c70004ba0 con 0x7f2c74073bd0 2024-09-11T07:56:12.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.742+0000 7f2c697fa640 1 -- 172.21.15.102:0/3994405499 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f2c70006030 con 0x7f2c74073bd0 2024-09-11T07:56:12.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.742+0000 7f2c697fa640 1 -- 172.21.15.102:0/3994405499 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 8) v1 ==== 45457+0+0 (unknown 296826312 0 0) 0x7f2c70011350 con 0x7f2c74073bd0 2024-09-11T07:56:12.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.742+0000 7f2c7b24f640 1 -- 172.21.15.102:0/3994405499 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c60005040 con 0x7f2c74073bd0 2024-09-11T07:56:12.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.742+0000 7f2c697fa640 1 -- 172.21.15.102:0/3994405499 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 1922949975 0 0) 0x7f2c70047940 con 0x7f2c74073bd0 2024-09-11T07:56:12.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.747+0000 7f2c697fa640 1 -- 172.21.15.102:0/3994405499 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f2c70012b80 con 0x7f2c74073bd0 2024-09-11T07:56:12.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.913+0000 7f2c7b24f640 1 -- 172.21.15.102:0/3994405499 --> v1:172.21.15.102:6800/3294079105 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2c60000bc0 con 0x7f2c5c03a6a0 2024-09-11T07:56:12.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.920+0000 7f2c697fa640 1 -- 172.21.15.102:0/3994405499 <== mgr.14118 v1:172.21.15.102:6800/3294079105 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 3265049985) 0x7f2c60000bc0 con 0x7f2c5c03a6a0 2024-09-11T07:56:12.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.922+0000 7f2c7b24f640 1 -- 172.21.15.102:0/3994405499 >> v1:172.21.15.102:6800/3294079105 conn(0x7f2c5c03a6a0 legacy=0x7f2c5c03cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:12.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.922+0000 7f2c7b24f640 1 -- 172.21.15.102:0/3994405499 >> v1:172.21.15.102:6789/0 conn(0x7f2c74073bd0 legacy=0x7f2c7410ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:12.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.922+0000 7f2c7b24f640 1 -- 172.21.15.102:0/3994405499 shutdown_connections 2024-09-11T07:56:12.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.922+0000 7f2c7b24f640 1 -- 172.21.15.102:0/3994405499 >> 172.21.15.102:0/3994405499 conn(0x7f2c740fa9d0 msgr2=0x7f2c74105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:12.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.922+0000 7f2c7b24f640 1 -- 172.21.15.102:0/3994405499 shutdown_connections 2024-09-11T07:56:12.618 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:11.922+0000 7f2c7b24f640 1 -- 172.21.15.102:0/3994405499 wait complete. 2024-09-11T07:56:12.619 INFO:teuthology.orchestra.run.smithi102.stdout:Deploying unmanaged mgr service... 2024-09-11T07:56:13.208 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:12 smithi102 ceph-mon[29849]: from='client.14140 v1:172.21.15.102:0/3994405499' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:56:13.208 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:12 smithi102 ceph-mon[29849]: Saving service mon spec with placement count:5 2024-09-11T07:56:13.209 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:12 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' 2024-09-11T07:56:14.527 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-11T07:56:14.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.548+0000 7f6c0bd9d640 1 Processor -- start 2024-09-11T07:56:14.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.548+0000 7f6c0bd9d640 1 -- start start 2024-09-11T07:56:14.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.549+0000 7f6c0bd9d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6c040ffd70 con 0x7f6c040ff250 2024-09-11T07:56:14.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.549+0000 7f6c09b39640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f6c040ff250 0x7f6c040ff660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46466/0 (socket says 172.21.15.102:46466) 2024-09-11T07:56:14.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.549+0000 7f6c09b39640 1 -- 172.21.15.102:0/3069071658 learned_addr learned my addr 172.21.15.102:0/3069071658 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:14.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.549+0000 7f6c08b37640 1 -- 172.21.15.102:0/3069071658 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3233951289 0 0) 0x7f6c040ffd70 con 0x7f6c040ff250 2024-09-11T07:56:14.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.549+0000 7f6c08b37640 1 -- 172.21.15.102:0/3069071658 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bf00034e0 con 0x7f6c040ff250 2024-09-11T07:56:14.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.549+0000 7f6c08b37640 1 -- 172.21.15.102:0/3069071658 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 790074382 0 0) 0x7f6bf00034e0 con 0x7f6c040ff250 2024-09-11T07:56:14.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.549+0000 7f6c08b37640 1 -- 172.21.15.102:0/3069071658 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c04104710 con 0x7f6c040ff250 2024-09-11T07:56:14.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.549+0000 7f6c08b37640 1 -- 172.21.15.102:0/3069071658 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f6c00001590 con 0x7f6c040ff250 2024-09-11T07:56:14.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.549+0000 7f6c08b37640 1 -- 172.21.15.102:0/3069071658 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7f6c00001cb0 con 0x7f6c040ff250 2024-09-11T07:56:14.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.549+0000 7f6c08b37640 1 -- 172.21.15.102:0/3069071658 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f6c00005670 con 0x7f6c040ff250 2024-09-11T07:56:14.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.550+0000 7f6c0bd9d640 1 -- 172.21.15.102:0/3069071658 >> v1:172.21.15.102:6789/0 conn(0x7f6c040ff250 legacy=0x7f6c040ff660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:14.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.550+0000 7f6c0bd9d640 1 -- 172.21.15.102:0/3069071658 shutdown_connections 2024-09-11T07:56:14.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.550+0000 7f6c0bd9d640 1 -- 172.21.15.102:0/3069071658 >> 172.21.15.102:0/3069071658 conn(0x7f6c040fa9d0 msgr2=0x7f6c040fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:14.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.550+0000 7f6c0bd9d640 1 -- 172.21.15.102:0/3069071658 shutdown_connections 2024-09-11T07:56:14.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.550+0000 7f6c0bd9d640 1 -- 172.21.15.102:0/3069071658 wait complete. 2024-09-11T07:56:14.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.550+0000 7f6c0bd9d640 1 Processor -- start 2024-09-11T07:56:14.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.550+0000 7f6c0bd9d640 1 -- start start 2024-09-11T07:56:14.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.551+0000 7f6c0bd9d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6c04112a00 con 0x7f6c040ff250 2024-09-11T07:56:14.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.551+0000 7f6c09b39640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f6c040ff250 0x7f6c04073960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46468/0 (socket says 172.21.15.102:46468) 2024-09-11T07:56:14.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.551+0000 7f6c09b39640 1 -- 172.21.15.102:0/3588608998 learned_addr learned my addr 172.21.15.102:0/3588608998 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:14.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.551+0000 7f6bfa7fc640 1 -- 172.21.15.102:0/3588608998 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2600703190 0 0) 0x7f6c04112a00 con 0x7f6c040ff250 2024-09-11T07:56:14.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.551+0000 7f6bfa7fc640 1 -- 172.21.15.102:0/3588608998 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bf00053f0 con 0x7f6c040ff250 2024-09-11T07:56:14.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.551+0000 7f6bfa7fc640 1 -- 172.21.15.102:0/3588608998 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2042914311 0 0) 0x7f6bf00053f0 con 0x7f6c040ff250 2024-09-11T07:56:14.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.551+0000 7f6bfa7fc640 1 -- 172.21.15.102:0/3588608998 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6c04112a00 con 0x7f6c040ff250 2024-09-11T07:56:14.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.551+0000 7f6bfa7fc640 1 -- 172.21.15.102:0/3588608998 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f6c000047a0 con 0x7f6c040ff250 2024-09-11T07:56:14.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.551+0000 7f6bfa7fc640 1 -- 172.21.15.102:0/3588608998 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4249416098 0 0) 0x7f6c04112a00 con 0x7f6c040ff250 2024-09-11T07:56:14.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.551+0000 7f6bfa7fc640 1 -- 172.21.15.102:0/3588608998 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c04113ba0 con 0x7f6c040ff250 2024-09-11T07:56:14.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.551+0000 7f6c0bd9d640 1 -- 172.21.15.102:0/3588608998 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c04112b90 con 0x7f6c040ff250 2024-09-11T07:56:14.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.551+0000 7f6c0bd9d640 1 -- 172.21.15.102:0/3588608998 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c04113060 con 0x7f6c040ff250 2024-09-11T07:56:14.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.551+0000 7f6bfa7fc640 1 -- 172.21.15.102:0/3588608998 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 1501596649 0 0) 0x7f6c00004ba0 con 0x7f6c040ff250 2024-09-11T07:56:14.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.551+0000 7f6bfa7fc640 1 -- 172.21.15.102:0/3588608998 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f6c00006110 con 0x7f6c040ff250 2024-09-11T07:56:14.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.552+0000 7f6bfa7fc640 1 -- 172.21.15.102:0/3588608998 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 8) v1 ==== 45457+0+0 (unknown 296826312 0 0) 0x7f6c000114d0 con 0x7f6c040ff250 2024-09-11T07:56:14.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.552+0000 7f6c0bd9d640 1 -- 172.21.15.102:0/3588608998 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be8005040 con 0x7f6c040ff250 2024-09-11T07:56:14.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.552+0000 7f6bfa7fc640 1 -- 172.21.15.102:0/3588608998 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 1922949975 0 0) 0x7f6c00047d70 con 0x7f6c040ff250 2024-09-11T07:56:14.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.557+0000 7f6bfa7fc640 1 -- 172.21.15.102:0/3588608998 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f6c00016c80 con 0x7f6c040ff250 2024-09-11T07:56:14.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.725+0000 7f6c0bd9d640 1 -- 172.21.15.102:0/3588608998 --> v1:172.21.15.102:6800/3294079105 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f6be8000bc0 con 0x7f6bf003a600 2024-09-11T07:56:14.531 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.729+0000 7f6bfa7fc640 1 -- 172.21.15.102:0/3588608998 <== mgr.14118 v1:172.21.15.102:6800/3294079105 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 325935098) 0x7f6be8000bc0 con 0x7f6bf003a600 2024-09-11T07:56:14.531 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.731+0000 7f6c0bd9d640 1 -- 172.21.15.102:0/3588608998 >> v1:172.21.15.102:6800/3294079105 conn(0x7f6bf003a600 legacy=0x7f6bf003cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:14.531 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.731+0000 7f6c0bd9d640 1 -- 172.21.15.102:0/3588608998 >> v1:172.21.15.102:6789/0 conn(0x7f6c040ff250 legacy=0x7f6c04073960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:14.531 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.731+0000 7f6c0bd9d640 1 -- 172.21.15.102:0/3588608998 shutdown_connections 2024-09-11T07:56:14.531 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.731+0000 7f6c0bd9d640 1 -- 172.21.15.102:0/3588608998 >> 172.21.15.102:0/3588608998 conn(0x7f6c040fa9d0 msgr2=0x7f6c04104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:14.531 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.731+0000 7f6c0bd9d640 1 -- 172.21.15.102:0/3588608998 shutdown_connections 2024-09-11T07:56:14.531 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:13.731+0000 7f6c0bd9d640 1 -- 172.21.15.102:0/3588608998 wait complete. 2024-09-11T07:56:14.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:14 smithi102 ceph-mon[29849]: from='client.14142 v1:172.21.15.102:0/3588608998' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:56:14.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:14 smithi102 ceph-mon[29849]: Saving service mgr spec with placement count:2 2024-09-11T07:56:14.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:14 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' 2024-09-11T07:56:14.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:14 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' 2024-09-11T07:56:14.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:14 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' 2024-09-11T07:56:16.423 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1863558186' entity='client.admin' 2024-09-11T07:56:16.424 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:16 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' 2024-09-11T07:56:16.431 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.975+0000 7f5b99fed640 1 Processor -- start 2024-09-11T07:56:16.431 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.975+0000 7f5b99fed640 1 -- start start 2024-09-11T07:56:16.431 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.975+0000 7f5b99fed640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5b94100f10 con 0x7f5b941003f0 2024-09-11T07:56:16.431 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.975+0000 7f5b937fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f5b941003f0 0x7f5b94100800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46480/0 (socket says 172.21.15.102:46480) 2024-09-11T07:56:16.431 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.975+0000 7f5b937fe640 1 -- 172.21.15.102:0/2999844945 learned_addr learned my addr 172.21.15.102:0/2999844945 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:16.431 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.975+0000 7f5b927fc640 1 -- 172.21.15.102:0/2999844945 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3497587265 0 0) 0x7f5b94100f10 con 0x7f5b941003f0 2024-09-11T07:56:16.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.975+0000 7f5b927fc640 1 -- 172.21.15.102:0/2999844945 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b7c0034e0 con 0x7f5b941003f0 2024-09-11T07:56:16.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.975+0000 7f5b927fc640 1 -- 172.21.15.102:0/2999844945 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1427631652 0 0) 0x7f5b7c0034e0 con 0x7f5b941003f0 2024-09-11T07:56:16.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.975+0000 7f5b927fc640 1 -- 172.21.15.102:0/2999844945 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b94105900 con 0x7f5b941003f0 2024-09-11T07:56:16.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.975+0000 7f5b927fc640 1 -- 172.21.15.102:0/2999844945 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f5b84001590 con 0x7f5b941003f0 2024-09-11T07:56:16.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.976+0000 7f5b927fc640 1 -- 172.21.15.102:0/2999844945 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f5b84001cd0 con 0x7f5b941003f0 2024-09-11T07:56:16.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.976+0000 7f5b927fc640 1 -- 172.21.15.102:0/2999844945 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f5b840056c0 con 0x7f5b941003f0 2024-09-11T07:56:16.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.976+0000 7f5b99fed640 1 -- 172.21.15.102:0/2999844945 >> v1:172.21.15.102:6789/0 conn(0x7f5b941003f0 legacy=0x7f5b94100800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:16.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.976+0000 7f5b99fed640 1 -- 172.21.15.102:0/2999844945 shutdown_connections 2024-09-11T07:56:16.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.976+0000 7f5b99fed640 1 -- 172.21.15.102:0/2999844945 >> 172.21.15.102:0/2999844945 conn(0x7f5b940fbb60 msgr2=0x7f5b940fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:16.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.976+0000 7f5b99fed640 1 -- 172.21.15.102:0/2999844945 shutdown_connections 2024-09-11T07:56:16.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.976+0000 7f5b99fed640 1 -- 172.21.15.102:0/2999844945 wait complete. 2024-09-11T07:56:16.432 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.977+0000 7f5b99fed640 1 Processor -- start 2024-09-11T07:56:16.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.977+0000 7f5b99fed640 1 -- start start 2024-09-11T07:56:16.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.977+0000 7f5b99fed640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5b9410f310 con 0x7f5b941003f0 2024-09-11T07:56:16.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.977+0000 7f5b937fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f5b941003f0 0x7f5b9410ec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46496/0 (socket says 172.21.15.102:46496) 2024-09-11T07:56:16.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.977+0000 7f5b937fe640 1 -- 172.21.15.102:0/1863558186 learned_addr learned my addr 172.21.15.102:0/1863558186 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:16.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.977+0000 7f5b7bfff640 1 -- 172.21.15.102:0/1863558186 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 35815186 0 0) 0x7f5b9410f310 con 0x7f5b941003f0 2024-09-11T07:56:16.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.977+0000 7f5b7bfff640 1 -- 172.21.15.102:0/1863558186 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b7c0053f0 con 0x7f5b941003f0 2024-09-11T07:56:16.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.977+0000 7f5b7bfff640 1 -- 172.21.15.102:0/1863558186 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4286695592 0 0) 0x7f5b7c0053f0 con 0x7f5b941003f0 2024-09-11T07:56:16.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.977+0000 7f5b7bfff640 1 -- 172.21.15.102:0/1863558186 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5b9410f310 con 0x7f5b941003f0 2024-09-11T07:56:16.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.977+0000 7f5b7bfff640 1 -- 172.21.15.102:0/1863558186 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f5b84001430 con 0x7f5b941003f0 2024-09-11T07:56:16.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.978+0000 7f5b7bfff640 1 -- 172.21.15.102:0/1863558186 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2586986864 0 0) 0x7f5b9410f310 con 0x7f5b941003f0 2024-09-11T07:56:16.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.978+0000 7f5b7bfff640 1 -- 172.21.15.102:0/1863558186 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9410f4a0 con 0x7f5b941003f0 2024-09-11T07:56:16.433 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.978+0000 7f5b99fed640 1 -- 172.21.15.102:0/1863558186 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b9410f770 con 0x7f5b941003f0 2024-09-11T07:56:16.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.978+0000 7f5b99fed640 1 -- 172.21.15.102:0/1863558186 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b94113250 con 0x7f5b941003f0 2024-09-11T07:56:16.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.978+0000 7f5b7bfff640 1 -- 172.21.15.102:0/1863558186 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f5b84001b30 con 0x7f5b941003f0 2024-09-11T07:56:16.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.978+0000 7f5b7bfff640 1 -- 172.21.15.102:0/1863558186 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f5b840061c0 con 0x7f5b941003f0 2024-09-11T07:56:16.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.978+0000 7f5b7bfff640 1 -- 172.21.15.102:0/1863558186 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 8) v1 ==== 45457+0+0 (unknown 296826312 0 0) 0x7f5b840114c0 con 0x7f5b941003f0 2024-09-11T07:56:16.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.979+0000 7f5b99fed640 1 -- 172.21.15.102:0/1863558186 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b80005040 con 0x7f5b941003f0 2024-09-11T07:56:16.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.979+0000 7f5b7bfff640 1 -- 172.21.15.102:0/1863558186 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 1922949975 0 0) 0x7f5b84047a90 con 0x7f5b941003f0 2024-09-11T07:56:16.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:14.983+0000 7f5b7bfff640 1 -- 172.21.15.102:0/1863558186 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f5b84016c50 con 0x7f5b941003f0 2024-09-11T07:56:16.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:15.178+0000 7f5b99fed640 1 -- 172.21.15.102:0/1863558186 --> v1:172.21.15.102:6789/0 -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f5b800052f0 con 0x7f5b941003f0 2024-09-11T07:56:16.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:15.181+0000 7f5b7bfff640 1 -- 172.21.15.102:0/1863558186 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (unknown 2684808024 0 0) 0x7f5b84016650 con 0x7f5b941003f0 2024-09-11T07:56:16.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:15.183+0000 7f5b99fed640 1 -- 172.21.15.102:0/1863558186 >> v1:172.21.15.102:6800/3294079105 conn(0x7f5b7c03a610 legacy=0x7f5b7c03cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:16.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:15.183+0000 7f5b99fed640 1 -- 172.21.15.102:0/1863558186 >> v1:172.21.15.102:6789/0 conn(0x7f5b941003f0 legacy=0x7f5b9410ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:16.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:15.183+0000 7f5b99fed640 1 -- 172.21.15.102:0/1863558186 shutdown_connections 2024-09-11T07:56:16.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:15.184+0000 7f5b99fed640 1 -- 172.21.15.102:0/1863558186 >> 172.21.15.102:0/1863558186 conn(0x7f5b940fbb60 msgr2=0x7f5b94105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:16.434 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:15.184+0000 7f5b99fed640 1 -- 172.21.15.102:0/1863558186 shutdown_connections 2024-09-11T07:56:16.435 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:15.184+0000 7f5b99fed640 1 -- 172.21.15.102:0/1863558186 wait complete. 2024-09-11T07:56:17.596 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.865+0000 7fa0d2cda640 1 Processor -- start 2024-09-11T07:56:17.596 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.865+0000 7fa0d2cda640 1 -- start start 2024-09-11T07:56:17.596 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.865+0000 7fa0d2cda640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa0cc0ffdd0 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.596 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.865+0000 7fa0d0a76640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fa0cc0ff2b0 0x7fa0cc0ff6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46498/0 (socket says 172.21.15.102:46498) 2024-09-11T07:56:17.596 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.865+0000 7fa0d0a76640 1 -- 172.21.15.102:0/4278968829 learned_addr learned my addr 172.21.15.102:0/4278968829 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:17.596 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.866+0000 7fa0cb7fe640 1 -- 172.21.15.102:0/4278968829 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2380058528 0 0) 0x7fa0cc0ffdd0 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.596 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.866+0000 7fa0cb7fe640 1 -- 172.21.15.102:0/4278968829 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0bc0034e0 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.596 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.866+0000 7fa0cb7fe640 1 -- 172.21.15.102:0/4278968829 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1469743612 0 0) 0x7fa0bc0034e0 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.596 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.866+0000 7fa0cb7fe640 1 -- 172.21.15.102:0/4278968829 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc104720 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.597 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.866+0000 7fa0cb7fe640 1 -- 172.21.15.102:0/4278968829 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fa0c0001590 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.597 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.866+0000 7fa0cb7fe640 1 -- 172.21.15.102:0/4278968829 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7fa0c0001cd0 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.597 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.866+0000 7fa0cb7fe640 1 -- 172.21.15.102:0/4278968829 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fa0c00056c0 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.597 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.867+0000 7fa0d2cda640 1 -- 172.21.15.102:0/4278968829 >> v1:172.21.15.102:6789/0 conn(0x7fa0cc0ff2b0 legacy=0x7fa0cc0ff6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:17.597 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.867+0000 7fa0d2cda640 1 -- 172.21.15.102:0/4278968829 shutdown_connections 2024-09-11T07:56:17.597 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.867+0000 7fa0d2cda640 1 -- 172.21.15.102:0/4278968829 >> 172.21.15.102:0/4278968829 conn(0x7fa0cc0faab0 msgr2=0x7fa0cc0fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:17.597 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.867+0000 7fa0d2cda640 1 -- 172.21.15.102:0/4278968829 shutdown_connections 2024-09-11T07:56:17.597 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.867+0000 7fa0d2cda640 1 -- 172.21.15.102:0/4278968829 wait complete. 2024-09-11T07:56:17.597 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.867+0000 7fa0d2cda640 1 Processor -- start 2024-09-11T07:56:17.597 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.867+0000 7fa0d2cda640 1 -- start start 2024-09-11T07:56:17.597 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.868+0000 7fa0d2cda640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa0cc10d100 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.598 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.868+0000 7fa0d0a76640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fa0cc0ff2b0 0x7fa0cc10c9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46502/0 (socket says 172.21.15.102:46502) 2024-09-11T07:56:17.598 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.868+0000 7fa0d0a76640 1 -- 172.21.15.102:0/1404897682 learned_addr learned my addr 172.21.15.102:0/1404897682 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:17.598 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.868+0000 7fa0c97fa640 1 -- 172.21.15.102:0/1404897682 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983257580 0 0) 0x7fa0cc10d100 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.598 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.868+0000 7fa0c97fa640 1 -- 172.21.15.102:0/1404897682 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0bc0053f0 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.598 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.868+0000 7fa0c97fa640 1 -- 172.21.15.102:0/1404897682 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1244568910 0 0) 0x7fa0bc0053f0 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.598 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.868+0000 7fa0c97fa640 1 -- 172.21.15.102:0/1404897682 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa0cc10d100 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.598 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.868+0000 7fa0c97fa640 1 -- 172.21.15.102:0/1404897682 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fa0c0001430 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.598 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.868+0000 7fa0c97fa640 1 -- 172.21.15.102:0/1404897682 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2570667669 0 0) 0x7fa0cc10d100 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.598 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.868+0000 7fa0c97fa640 1 -- 172.21.15.102:0/1404897682 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc10d290 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.598 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.868+0000 7fa0d2cda640 1 -- 172.21.15.102:0/1404897682 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0cc10d560 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.598 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.868+0000 7fa0d2cda640 1 -- 172.21.15.102:0/1404897682 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0cc111040 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.598 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.868+0000 7fa0c97fa640 1 -- 172.21.15.102:0/1404897682 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7fa0c0001b30 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.599 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.868+0000 7fa0c97fa640 1 -- 172.21.15.102:0/1404897682 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fa0c0006180 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.599 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.869+0000 7fa0c97fa640 1 -- 172.21.15.102:0/1404897682 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 8) v1 ==== 45457+0+0 (unknown 296826312 0 0) 0x7fa0c0011500 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.599 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.869+0000 7fa0d2cda640 1 -- 172.21.15.102:0/1404897682 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b0005040 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.599 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.869+0000 7fa0c97fa640 1 -- 172.21.15.102:0/1404897682 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 1922949975 0 0) 0x7fa0c0047b30 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.599 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:16.874+0000 7fa0c97fa640 1 -- 172.21.15.102:0/1404897682 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fa0c0016d10 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.599 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:17.039+0000 7fa0d2cda640 1 -- 172.21.15.102:0/1404897682 --> v1:172.21.15.102:6789/0 -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fa0b0005870 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.599 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:17.042+0000 7fa0c97fa640 1 -- 172.21.15.102:0/1404897682 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (unknown 1496729260 0 0) 0x7fa0c0016710 con 0x7fa0cc0ff2b0 2024-09-11T07:56:17.599 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:17.044+0000 7fa0d2cda640 1 -- 172.21.15.102:0/1404897682 >> v1:172.21.15.102:6800/3294079105 conn(0x7fa0bc03c7d0 legacy=0x7fa0bc03ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:17.599 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:17.044+0000 7fa0d2cda640 1 -- 172.21.15.102:0/1404897682 >> v1:172.21.15.102:6789/0 conn(0x7fa0cc0ff2b0 legacy=0x7fa0cc10c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:17.599 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:17.044+0000 7fa0d2cda640 1 -- 172.21.15.102:0/1404897682 shutdown_connections 2024-09-11T07:56:17.599 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:17.044+0000 7fa0d2cda640 1 -- 172.21.15.102:0/1404897682 >> 172.21.15.102:0/1404897682 conn(0x7fa0cc0faab0 msgr2=0x7fa0cc0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:17.600 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:17.044+0000 7fa0d2cda640 1 -- 172.21.15.102:0/1404897682 shutdown_connections 2024-09-11T07:56:17.600 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:17.044+0000 7fa0d2cda640 1 -- 172.21.15.102:0/1404897682 wait complete. 2024-09-11T07:56:17.600 INFO:teuthology.orchestra.run.smithi102.stdout:Enabling the dashboard module... 2024-09-11T07:56:18.149 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:17 smithi102 ceph-mon[29849]: from='mgr.14118 v1:172.21.15.102:0/2610777489' entity='mgr.y' 2024-09-11T07:56:18.149 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1404897682' entity='client.admin' 2024-09-11T07:56:19.106 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:18 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ignoring --setuser ceph since I am not root 2024-09-11T07:56:19.106 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:18 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ignoring --setgroup ceph since I am not root 2024-09-11T07:56:19.107 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:19.057+0000 7f4d5a2a5ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:56:19.107 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2834441940' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-11T07:56:19.396 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:19.112+0000 7f4d5a2a5ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:56:19.396 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:19.179+0000 7f4d5a2a5ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:56:19.396 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:19.302+0000 7f4d5a2a5ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:56:19.701 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:19.441+0000 7f4d5a2a5ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:56:19.701 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:19.699+0000 7f4d5a2a5ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:56:19.750 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.024+0000 7fcd53046640 1 Processor -- start 2024-09-11T07:56:19.750 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.024+0000 7fcd53046640 1 -- start start 2024-09-11T07:56:19.750 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.024+0000 7fcd53046640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcd4c100e40 con 0x7fcd4c100320 2024-09-11T07:56:19.750 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.024+0000 7fcd50de2640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fcd4c100320 0x7fcd4c100730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38382/0 (socket says 172.21.15.102:38382) 2024-09-11T07:56:19.750 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.024+0000 7fcd50de2640 1 -- 172.21.15.102:0/274868830 learned_addr learned my addr 172.21.15.102:0/274868830 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:19.750 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.024+0000 7fcd4b7fe640 1 -- 172.21.15.102:0/274868830 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034867414 0 0) 0x7fcd4c100e40 con 0x7fcd4c100320 2024-09-11T07:56:19.750 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.024+0000 7fcd4b7fe640 1 -- 172.21.15.102:0/274868830 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd380034e0 con 0x7fcd4c100320 2024-09-11T07:56:19.750 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.025+0000 7fcd4b7fe640 1 -- 172.21.15.102:0/274868830 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1234562010 0 0) 0x7fcd380034e0 con 0x7fcd4c100320 2024-09-11T07:56:19.750 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.025+0000 7fcd4b7fe640 1 -- 172.21.15.102:0/274868830 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4c105830 con 0x7fcd4c100320 2024-09-11T07:56:19.751 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.025+0000 7fcd4b7fe640 1 -- 172.21.15.102:0/274868830 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fcd44001590 con 0x7fcd4c100320 2024-09-11T07:56:19.751 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.025+0000 7fcd4b7fe640 1 -- 172.21.15.102:0/274868830 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7fcd44001cd0 con 0x7fcd4c100320 2024-09-11T07:56:19.751 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.025+0000 7fcd4b7fe640 1 -- 172.21.15.102:0/274868830 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fcd440056c0 con 0x7fcd4c100320 2024-09-11T07:56:19.751 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.025+0000 7fcd53046640 1 -- 172.21.15.102:0/274868830 >> v1:172.21.15.102:6789/0 conn(0x7fcd4c100320 legacy=0x7fcd4c100730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:19.751 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.025+0000 7fcd53046640 1 -- 172.21.15.102:0/274868830 shutdown_connections 2024-09-11T07:56:19.751 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.025+0000 7fcd53046640 1 -- 172.21.15.102:0/274868830 >> 172.21.15.102:0/274868830 conn(0x7fcd4c0fbb60 msgr2=0x7fcd4c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:19.751 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.025+0000 7fcd53046640 1 -- 172.21.15.102:0/274868830 shutdown_connections 2024-09-11T07:56:19.751 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.025+0000 7fcd53046640 1 -- 172.21.15.102:0/274868830 wait complete. 2024-09-11T07:56:19.751 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.026+0000 7fcd53046640 1 Processor -- start 2024-09-11T07:56:19.752 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.026+0000 7fcd53046640 1 -- start start 2024-09-11T07:56:19.752 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.026+0000 7fcd53046640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcd4c10f370 con 0x7fcd4c100320 2024-09-11T07:56:19.752 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.026+0000 7fcd50de2640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fcd4c100320 0x7fcd4c10ec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38384/0 (socket says 172.21.15.102:38384) 2024-09-11T07:56:19.752 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.026+0000 7fcd50de2640 1 -- 172.21.15.102:0/2834441940 learned_addr learned my addr 172.21.15.102:0/2834441940 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:19.752 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.026+0000 7fcd497fa640 1 -- 172.21.15.102:0/2834441940 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2285612582 0 0) 0x7fcd4c10f370 con 0x7fcd4c100320 2024-09-11T07:56:19.752 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.026+0000 7fcd497fa640 1 -- 172.21.15.102:0/2834441940 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd380053f0 con 0x7fcd4c100320 2024-09-11T07:56:19.752 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.027+0000 7fcd497fa640 1 -- 172.21.15.102:0/2834441940 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3662359550 0 0) 0x7fcd380053f0 con 0x7fcd4c100320 2024-09-11T07:56:19.752 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.027+0000 7fcd497fa640 1 -- 172.21.15.102:0/2834441940 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcd4c10f370 con 0x7fcd4c100320 2024-09-11T07:56:19.753 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.027+0000 7fcd497fa640 1 -- 172.21.15.102:0/2834441940 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fcd44001430 con 0x7fcd4c100320 2024-09-11T07:56:19.753 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.027+0000 7fcd497fa640 1 -- 172.21.15.102:0/2834441940 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1387652296 0 0) 0x7fcd4c10f370 con 0x7fcd4c100320 2024-09-11T07:56:19.753 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.027+0000 7fcd497fa640 1 -- 172.21.15.102:0/2834441940 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4c10f500 con 0x7fcd4c100320 2024-09-11T07:56:19.753 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.027+0000 7fcd53046640 1 -- 172.21.15.102:0/2834441940 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd4c10f7d0 con 0x7fcd4c100320 2024-09-11T07:56:19.753 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.027+0000 7fcd53046640 1 -- 172.21.15.102:0/2834441940 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd4c113260 con 0x7fcd4c100320 2024-09-11T07:56:19.753 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.027+0000 7fcd497fa640 1 -- 172.21.15.102:0/2834441940 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7fcd44001b30 con 0x7fcd4c100320 2024-09-11T07:56:19.753 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.027+0000 7fcd497fa640 1 -- 172.21.15.102:0/2834441940 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fcd440062a0 con 0x7fcd4c100320 2024-09-11T07:56:19.753 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.028+0000 7fcd497fa640 1 -- 172.21.15.102:0/2834441940 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 8) v1 ==== 45457+0+0 (unknown 296826312 0 0) 0x7fcd44006510 con 0x7fcd4c100320 2024-09-11T07:56:19.753 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.028+0000 7fcd497fa640 1 -- 172.21.15.102:0/2834441940 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 1922949975 0 0) 0x7fcd44047ba0 con 0x7fcd4c100320 2024-09-11T07:56:19.753 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.028+0000 7fcd53046640 1 -- 172.21.15.102:0/2834441940 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd2c005040 con 0x7fcd4c100320 2024-09-11T07:56:19.753 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.032+0000 7fcd497fa640 1 -- 172.21.15.102:0/2834441940 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fcd44016d50 con 0x7fcd4c100320 2024-09-11T07:56:19.753 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.246+0000 7fcd53046640 1 -- 172.21.15.102:0/2834441940 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fcd2c0052f0 con 0x7fcd4c100320 2024-09-11T07:56:19.753 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.919+0000 7fcd497fa640 1 -- 172.21.15.102:0/2834441940 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mgrmap(e 9) v1 ==== 45470+0+0 (unknown 657574904 0 0) 0x7fcd44013760 con 0x7fcd4c100320 2024-09-11T07:56:19.754 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.921+0000 7fcd497fa640 1 -- 172.21.15.102:0/2834441940 <== mon.0 v1:172.21.15.102:6789/0 11 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (unknown 1498667528 0 0) 0x7fcd44016f80 con 0x7fcd4c100320 2024-09-11T07:56:19.754 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.923+0000 7fcd53046640 1 -- 172.21.15.102:0/2834441940 >> v1:172.21.15.102:6800/3294079105 conn(0x7fcd3803a5c0 legacy=0x7fcd3803ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:19.754 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.923+0000 7fcd53046640 1 -- 172.21.15.102:0/2834441940 >> v1:172.21.15.102:6789/0 conn(0x7fcd4c100320 legacy=0x7fcd4c10ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:19.754 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.923+0000 7fcd53046640 1 -- 172.21.15.102:0/2834441940 shutdown_connections 2024-09-11T07:56:19.754 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.923+0000 7fcd53046640 1 -- 172.21.15.102:0/2834441940 >> 172.21.15.102:0/2834441940 conn(0x7fcd4c0fbb60 msgr2=0x7fcd4c105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:19.754 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.923+0000 7fcd53046640 1 -- 172.21.15.102:0/2834441940 shutdown_connections 2024-09-11T07:56:19.754 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:18.923+0000 7fcd53046640 1 -- 172.21.15.102:0/2834441940 wait complete. 2024-09-11T07:56:19.976 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2834441940' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-11T07:56:19.976 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:19 smithi102 ceph-mon[29849]: mgrmap e9: y(active, since 17s) 2024-09-11T07:56:19.977 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:19.762+0000 7f4d5a2a5ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:56:19.977 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:19.930+0000 7f4d5a2a5ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:56:21.027 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3076988122' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T07:56:21.028 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:20 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:20.805+0000 7f4d5a2a5ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:56:21.028 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:20 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:20.870+0000 7f4d5a2a5ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:56:21.033 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:56:21.033 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-11T07:56:21.033 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T07:56:21.033 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-11T07:56:21.033 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T07:56:21.033 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:56:21.033 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.179+0000 7f8a2d361640 1 Processor -- start 2024-09-11T07:56:21.033 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.180+0000 7f8a2d361640 1 -- start start 2024-09-11T07:56:21.033 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.180+0000 7f8a2d361640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8a281046a0 con 0x7f8a28074510 2024-09-11T07:56:21.034 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.180+0000 7f8a26ffd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f8a28074510 0x7f8a28102f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38426/0 (socket says 172.21.15.102:38426) 2024-09-11T07:56:21.034 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.180+0000 7f8a26ffd640 1 -- 172.21.15.102:0/954182027 learned_addr learned my addr 172.21.15.102:0/954182027 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:21.034 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.180+0000 7f8a25ffb640 1 -- 172.21.15.102:0/954182027 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1066290138 0 0) 0x7f8a281046a0 con 0x7f8a28074510 2024-09-11T07:56:21.034 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.180+0000 7f8a25ffb640 1 -- 172.21.15.102:0/954182027 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a100034e0 con 0x7f8a28074510 2024-09-11T07:56:21.034 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.180+0000 7f8a25ffb640 1 -- 172.21.15.102:0/954182027 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3248940535 0 0) 0x7f8a100034e0 con 0x7f8a28074510 2024-09-11T07:56:21.034 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.180+0000 7f8a25ffb640 1 -- 172.21.15.102:0/954182027 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a28105840 con 0x7f8a28074510 2024-09-11T07:56:21.034 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.180+0000 7f8a25ffb640 1 -- 172.21.15.102:0/954182027 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f8a18001590 con 0x7f8a28074510 2024-09-11T07:56:21.034 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.181+0000 7f8a25ffb640 1 -- 172.21.15.102:0/954182027 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f8a18001c00 con 0x7f8a28074510 2024-09-11T07:56:21.034 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.181+0000 7f8a25ffb640 1 -- 172.21.15.102:0/954182027 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f8a180055e0 con 0x7f8a28074510 2024-09-11T07:56:21.034 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.181+0000 7f8a2d361640 1 -- 172.21.15.102:0/954182027 >> v1:172.21.15.102:6789/0 conn(0x7f8a28074510 legacy=0x7f8a28102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:21.034 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.181+0000 7f8a2d361640 1 -- 172.21.15.102:0/954182027 shutdown_connections 2024-09-11T07:56:21.034 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.181+0000 7f8a2d361640 1 -- 172.21.15.102:0/954182027 >> 172.21.15.102:0/954182027 conn(0x7f8a280faa90 msgr2=0x7f8a280fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:21.034 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.181+0000 7f8a2d361640 1 -- 172.21.15.102:0/954182027 shutdown_connections 2024-09-11T07:56:21.034 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.181+0000 7f8a2d361640 1 -- 172.21.15.102:0/954182027 wait complete. 2024-09-11T07:56:21.035 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.182+0000 7f8a2d361640 1 Processor -- start 2024-09-11T07:56:21.035 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.182+0000 7f8a2d361640 1 -- start start 2024-09-11T07:56:21.035 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.182+0000 7f8a2d361640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8a2810f380 con 0x7f8a28074510 2024-09-11T07:56:21.035 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.182+0000 7f8a26ffd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f8a28074510 0x7f8a2810ec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38438/0 (socket says 172.21.15.102:38438) 2024-09-11T07:56:21.035 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.182+0000 7f8a26ffd640 1 -- 172.21.15.102:0/3076988122 learned_addr learned my addr 172.21.15.102:0/3076988122 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:21.035 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.182+0000 7f8a0f7fe640 1 -- 172.21.15.102:0/3076988122 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995552279 0 0) 0x7f8a2810f380 con 0x7f8a28074510 2024-09-11T07:56:21.035 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.182+0000 7f8a0f7fe640 1 -- 172.21.15.102:0/3076988122 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a10005440 con 0x7f8a28074510 2024-09-11T07:56:21.035 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.182+0000 7f8a0f7fe640 1 -- 172.21.15.102:0/3076988122 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3787412494 0 0) 0x7f8a10005440 con 0x7f8a28074510 2024-09-11T07:56:21.035 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.182+0000 7f8a0f7fe640 1 -- 172.21.15.102:0/3076988122 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8a2810f380 con 0x7f8a28074510 2024-09-11T07:56:21.035 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.182+0000 7f8a0f7fe640 1 -- 172.21.15.102:0/3076988122 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f8a180048c0 con 0x7f8a28074510 2024-09-11T07:56:21.035 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.183+0000 7f8a0f7fe640 1 -- 172.21.15.102:0/3076988122 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 480418456 0 0) 0x7f8a2810f380 con 0x7f8a28074510 2024-09-11T07:56:21.035 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.183+0000 7f8a0f7fe640 1 -- 172.21.15.102:0/3076988122 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2810f510 con 0x7f8a28074510 2024-09-11T07:56:21.035 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.183+0000 7f8a2d361640 1 -- 172.21.15.102:0/3076988122 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a2810f7e0 con 0x7f8a28074510 2024-09-11T07:56:21.036 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.183+0000 7f8a2d361640 1 -- 172.21.15.102:0/3076988122 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a281132c0 con 0x7f8a28074510 2024-09-11T07:56:21.036 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.183+0000 7f8a0f7fe640 1 -- 172.21.15.102:0/3076988122 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f8a18001800 con 0x7f8a28074510 2024-09-11T07:56:21.036 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.183+0000 7f8a0f7fe640 1 -- 172.21.15.102:0/3076988122 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f8a18006070 con 0x7f8a28074510 2024-09-11T07:56:21.036 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.184+0000 7f8a2d361640 1 -- 172.21.15.102:0/3076988122 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a14005040 con 0x7f8a28074510 2024-09-11T07:56:21.036 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.184+0000 7f8a0f7fe640 1 -- 172.21.15.102:0/3076988122 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 9) v1 ==== 45470+0+0 (unknown 657574904 0 0) 0x7f8a18011400 con 0x7f8a28074510 2024-09-11T07:56:21.036 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.184+0000 7f8a0f7fe640 1 -- 172.21.15.102:0/3076988122 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 1922949975 0 0) 0x7f8a18047b70 con 0x7f8a28074510 2024-09-11T07:56:21.036 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.184+0000 7f8a267fc640 1 -- 172.21.15.102:0/3076988122 >> v1:172.21.15.102:6800/3294079105 conn(0x7f8a1003a6f0 legacy=0x7f8a1003cba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.102:6800/3294079105 2024-09-11T07:56:21.036 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.188+0000 7f8a0f7fe640 1 -- 172.21.15.102:0/3076988122 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f8a18012c30 con 0x7f8a28074510 2024-09-11T07:56:21.036 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.385+0000 7f8a267fc640 1 -- 172.21.15.102:0/3076988122 >> v1:172.21.15.102:6800/3294079105 conn(0x7f8a1003a6f0 legacy=0x7f8a1003cba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.102:6800/3294079105 2024-09-11T07:56:21.036 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.392+0000 7f8a2d361640 1 -- 172.21.15.102:0/3076988122 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f8a14005b00 con 0x7f8a28074510 2024-09-11T07:56:21.036 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.392+0000 7f8a0f7fe640 1 -- 172.21.15.102:0/3076988122 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (unknown 2005831594 0 1748205903) 0x7f8a18016680 con 0x7f8a28074510 2024-09-11T07:56:21.036 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.394+0000 7f8a2d361640 1 -- 172.21.15.102:0/3076988122 >> v1:172.21.15.102:6800/3294079105 conn(0x7f8a1003a6f0 legacy=0x7f8a1003cba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T07:56:21.037 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.394+0000 7f8a2d361640 1 -- 172.21.15.102:0/3076988122 >> v1:172.21.15.102:6789/0 conn(0x7f8a28074510 legacy=0x7f8a2810ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:21.037 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.394+0000 7f8a2d361640 1 -- 172.21.15.102:0/3076988122 shutdown_connections 2024-09-11T07:56:21.037 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.394+0000 7f8a2d361640 1 -- 172.21.15.102:0/3076988122 >> 172.21.15.102:0/3076988122 conn(0x7f8a280faa90 msgr2=0x7f8a28102840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:21.037 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.394+0000 7f8a2d361640 1 -- 172.21.15.102:0/3076988122 shutdown_connections 2024-09-11T07:56:21.037 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:20.394+0000 7f8a2d361640 1 -- 172.21.15.102:0/3076988122 wait complete. 2024-09-11T07:56:21.037 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for the mgr to restart... 2024-09-11T07:56:21.037 INFO:teuthology.orchestra.run.smithi102.stdout:Waiting for mgr epoch 9... 2024-09-11T07:56:21.895 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:21 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:21.616+0000 7f4d5a2a5ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:56:21.896 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:21 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:21.728+0000 7f4d5a2a5ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:56:21.896 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:21 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:21.847+0000 7f4d5a2a5ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:56:22.287 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:21 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:21.905+0000 7f4d5a2a5ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:56:22.287 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:22 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:22.028+0000 7f4d5a2a5ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:56:22.287 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:22 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:22.095+0000 7f4d5a2a5ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:56:22.288 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:22 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:22.152+0000 7f4d5a2a5ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:56:22.288 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:22 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:22.284+0000 7f4d5a2a5ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:56:22.563 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:22 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:22.349+0000 7f4d5a2a5ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:56:22.564 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:22 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:22.417+0000 7f4d5a2a5ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:56:22.564 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:22 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:22.501+0000 7f4d5a2a5ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:56:22.564 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:22 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:22.560+0000 7f4d5a2a5ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:56:22.813 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:22 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:22.620+0000 7f4d5a2a5ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:56:23.896 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:23 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:23.397+0000 7f4d5a2a5ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:56:23.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:23 smithi102 ceph-mon[29849]: Active manager daemon y restarted 2024-09-11T07:56:23.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:23 smithi102 ceph-mon[29849]: Activating manager daemon y 2024-09-11T07:56:23.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:23 smithi102 ceph-mon[29849]: osdmap e3: 0 total, 0 up, 0 in 2024-09-11T07:56:23.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:23 smithi102 ceph-mon[29849]: mgrmap e10: y(active, starting, since 0.00384687s) 2024-09-11T07:56:23.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:23 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:56:23.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:23 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:56:23.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:23 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:56:23.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:23 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:56:23.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:23 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:56:23.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:23 smithi102 ceph-mon[29849]: Manager daemon y is now available 2024-09-11T07:56:23.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:23 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:23.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:23 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:56:23.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:23 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:56:24.886 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:24 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:24.757+0000 7f4d0ae02640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:07:56:24] ENGINE Error in HTTPServer.serve 2024-09-11T07:56:24.886 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:24 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: Traceback (most recent call last): 2024-09-11T07:56:24.886 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:24 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:56:24.886 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:24 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: self._connections.run(self.expiration_interval) 2024-09-11T07:56:24.886 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:24 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:56:24.886 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:24 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: self._run(expiration_interval) 2024-09-11T07:56:24.886 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:24 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:56:24.887 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:24 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:56:24.887 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:24 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:56:24.887 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:24 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:56:24.887 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:24 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:56:24.887 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:24 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: s = self.context.wrap_socket( 2024-09-11T07:56:24.887 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:24 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:56:24.887 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:24 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: return self.sslsocket_class._create( 2024-09-11T07:56:24.887 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:24 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:56:24.887 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:24 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: self.do_handshake() 2024-09-11T07:56:24.887 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:24 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:56:24.887 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:24 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: self._sslobj.do_handshake() 2024-09-11T07:56:24.887 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:24 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:56:24.887 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 07:56:24 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T07:56:25.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:56:25.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-11T07:56:25.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T07:56:25.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:56:25.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.520+0000 7ffbc29bf640 1 Processor -- start 2024-09-11T07:56:25.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.520+0000 7ffbc29bf640 1 -- start start 2024-09-11T07:56:25.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.521+0000 7ffbc29bf640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ffbbc100db0 con 0x7ffbbc073bf0 2024-09-11T07:56:25.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.521+0000 7ffbbbfff640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ffbbc073bf0 0x7ffbbc1053c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38454/0 (socket says 172.21.15.102:38454) 2024-09-11T07:56:25.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.521+0000 7ffbbbfff640 1 -- 172.21.15.102:0/3383641056 learned_addr learned my addr 172.21.15.102:0/3383641056 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:25.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.521+0000 7ffbbaffd640 1 -- 172.21.15.102:0/3383641056 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1340764722 0 0) 0x7ffbbc100db0 con 0x7ffbbc073bf0 2024-09-11T07:56:25.169 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.521+0000 7ffbbaffd640 1 -- 172.21.15.102:0/3383641056 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffba40034e0 con 0x7ffbbc073bf0 2024-09-11T07:56:25.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.521+0000 7ffbbaffd640 1 -- 172.21.15.102:0/3383641056 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1996835599 0 0) 0x7ffba40034e0 con 0x7ffbbc073bf0 2024-09-11T07:56:25.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.521+0000 7ffbbaffd640 1 -- 172.21.15.102:0/3383641056 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc107af0 con 0x7ffbbc073bf0 2024-09-11T07:56:25.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.521+0000 7ffbbaffd640 1 -- 172.21.15.102:0/3383641056 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7ffbac001590 con 0x7ffbbc073bf0 2024-09-11T07:56:25.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.521+0000 7ffbbaffd640 1 -- 172.21.15.102:0/3383641056 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7ffbac001cd0 con 0x7ffbbc073bf0 2024-09-11T07:56:25.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.521+0000 7ffbbaffd640 1 -- 172.21.15.102:0/3383641056 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7ffbac005650 con 0x7ffbbc073bf0 2024-09-11T07:56:25.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.522+0000 7ffbc29bf640 1 -- 172.21.15.102:0/3383641056 >> v1:172.21.15.102:6789/0 conn(0x7ffbbc073bf0 legacy=0x7ffbbc1053c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:25.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.522+0000 7ffbc29bf640 1 -- 172.21.15.102:0/3383641056 shutdown_connections 2024-09-11T07:56:25.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.522+0000 7ffbc29bf640 1 -- 172.21.15.102:0/3383641056 >> 172.21.15.102:0/3383641056 conn(0x7ffbbc0fa9d0 msgr2=0x7ffbbc0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:25.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.522+0000 7ffbc29bf640 1 -- 172.21.15.102:0/3383641056 shutdown_connections 2024-09-11T07:56:25.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.522+0000 7ffbc29bf640 1 -- 172.21.15.102:0/3383641056 wait complete. 2024-09-11T07:56:25.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.522+0000 7ffbc29bf640 1 Processor -- start 2024-09-11T07:56:25.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.523+0000 7ffbc29bf640 1 -- start start 2024-09-11T07:56:25.170 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.523+0000 7ffbc29bf640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ffbbc111480 con 0x7ffbbc073bf0 2024-09-11T07:56:25.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.523+0000 7ffbbbfff640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ffbbc073bf0 0x7ffbbc110d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38456/0 (socket says 172.21.15.102:38456) 2024-09-11T07:56:25.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.523+0000 7ffbbbfff640 1 -- 172.21.15.102:0/3865495964 learned_addr learned my addr 172.21.15.102:0/3865495964 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:25.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.523+0000 7ffbb8ff9640 1 -- 172.21.15.102:0/3865495964 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1080368031 0 0) 0x7ffbbc111480 con 0x7ffbbc073bf0 2024-09-11T07:56:25.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.523+0000 7ffbb8ff9640 1 -- 172.21.15.102:0/3865495964 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffba40053f0 con 0x7ffbbc073bf0 2024-09-11T07:56:25.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.523+0000 7ffbb8ff9640 1 -- 172.21.15.102:0/3865495964 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 564043760 0 0) 0x7ffba40053f0 con 0x7ffbbc073bf0 2024-09-11T07:56:25.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.523+0000 7ffbb8ff9640 1 -- 172.21.15.102:0/3865495964 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffbbc111480 con 0x7ffbbc073bf0 2024-09-11T07:56:25.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.523+0000 7ffbb8ff9640 1 -- 172.21.15.102:0/3865495964 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7ffbac005410 con 0x7ffbbc073bf0 2024-09-11T07:56:25.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.523+0000 7ffbb8ff9640 1 -- 172.21.15.102:0/3865495964 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3979516452 0 0) 0x7ffbbc111480 con 0x7ffbbc073bf0 2024-09-11T07:56:25.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.523+0000 7ffbb8ff9640 1 -- 172.21.15.102:0/3865495964 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc111610 con 0x7ffbbc073bf0 2024-09-11T07:56:25.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.523+0000 7ffbc29bf640 1 -- 172.21.15.102:0/3865495964 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbbc1118e0 con 0x7ffbbc073bf0 2024-09-11T07:56:25.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.524+0000 7ffbc29bf640 1 -- 172.21.15.102:0/3865495964 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbbc1153c0 con 0x7ffbbc073bf0 2024-09-11T07:56:25.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.524+0000 7ffbb8ff9640 1 -- 172.21.15.102:0/3865495964 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7ffbac004b60 con 0x7ffbbc073bf0 2024-09-11T07:56:25.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.524+0000 7ffbb8ff9640 1 -- 172.21.15.102:0/3865495964 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7ffbac0060e0 con 0x7ffbbc073bf0 2024-09-11T07:56:25.171 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.525+0000 7ffbb8ff9640 1 -- 172.21.15.102:0/3865495964 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 9) v1 ==== 45470+0+0 (unknown 657574904 0 0) 0x7ffbac0114b0 con 0x7ffbbc073bf0 2024-09-11T07:56:25.172 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.525+0000 7ffbb8ff9640 1 -- 172.21.15.102:0/3865495964 --> v1:172.21.15.102:6800/3294079105 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffba403e360 con 0x7ffba403a6a0 2024-09-11T07:56:25.172 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.525+0000 7ffbb8ff9640 1 -- 172.21.15.102:0/3865495964 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 1922949975 0 0) 0x7ffbac047c10 con 0x7ffbbc073bf0 2024-09-11T07:56:25.172 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.525+0000 7ffbbb7fe640 1 -- 172.21.15.102:0/3865495964 >> v1:172.21.15.102:6800/3294079105 conn(0x7ffba403a6a0 legacy=0x7ffba403cb50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.102:6800/3294079105 2024-09-11T07:56:25.172 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:21.726+0000 7ffbbb7fe640 1 -- 172.21.15.102:0/3865495964 >> v1:172.21.15.102:6800/3294079105 conn(0x7ffba403a6a0 legacy=0x7ffba403cb50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.102:6800/3294079105 2024-09-11T07:56:25.172 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:22.127+0000 7ffbbb7fe640 1 -- 172.21.15.102:0/3865495964 >> v1:172.21.15.102:6800/3294079105 conn(0x7ffba403a6a0 legacy=0x7ffba403cb50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.102:6800/3294079105 2024-09-11T07:56:25.172 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:22.928+0000 7ffbbb7fe640 1 -- 172.21.15.102:0/3865495964 >> v1:172.21.15.102:6800/3294079105 conn(0x7ffba403a6a0 legacy=0x7ffba403cb50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.102:6800/3294079105 2024-09-11T07:56:25.172 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:23.401+0000 7ffbb8ff9640 1 -- 172.21.15.102:0/3865495964 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mgrmap(e 10) v1 ==== 45272+0+0 (unknown 44003371 0 0) 0x7ffbac046c50 con 0x7ffbbc073bf0 2024-09-11T07:56:25.172 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:23.402+0000 7ffbb8ff9640 1 -- 172.21.15.102:0/3865495964 >> v1:172.21.15.102:6800/3294079105 conn(0x7ffba403a6a0 legacy=0x7ffba403cb50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T07:56:25.172 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:24.402+0000 7ffbb8ff9640 1 -- 172.21.15.102:0/3865495964 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mgrmap(e 11) v1 ==== 45364+0+0 (unknown 1346339249 0 0) 0x7ffbac0490a0 con 0x7ffbbc073bf0 2024-09-11T07:56:25.172 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:24.403+0000 7ffbb8ff9640 1 -- 172.21.15.102:0/3865495964 --> v1:172.21.15.102:6800/2950771867 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffba403e360 con 0x7ffba403efc0 2024-09-11T07:56:25.172 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:24.404+0000 7ffbb8ff9640 1 -- 172.21.15.102:0/3865495964 <== mgr.14150 v1:172.21.15.102:6800/2950771867 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (unknown 0 0 4193150809) 0x7ffba403e360 con 0x7ffba403efc0 2024-09-11T07:56:25.172 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:24.409+0000 7ffbc29bf640 1 -- 172.21.15.102:0/3865495964 --> v1:172.21.15.102:6800/2950771867 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ffbbc1075e0 con 0x7ffba403efc0 2024-09-11T07:56:25.172 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:24.409+0000 7ffbb8ff9640 1 -- 172.21.15.102:0/3865495964 <== mgr.14150 v1:172.21.15.102:6800/2950771867 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (unknown 0 0 3086460295) 0x7ffbbc1075e0 con 0x7ffba403efc0 2024-09-11T07:56:25.173 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:24.409+0000 7ffbc29bf640 1 -- 172.21.15.102:0/3865495964 >> v1:172.21.15.102:6800/2950771867 conn(0x7ffba403efc0 legacy=0x7ffba40413a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:25.173 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:24.410+0000 7ffbc29bf640 1 -- 172.21.15.102:0/3865495964 >> v1:172.21.15.102:6789/0 conn(0x7ffbbc073bf0 legacy=0x7ffbbc110d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:25.173 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:24.410+0000 7ffbc29bf640 1 -- 172.21.15.102:0/3865495964 shutdown_connections 2024-09-11T07:56:25.173 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:24.410+0000 7ffbc29bf640 1 -- 172.21.15.102:0/3865495964 >> 172.21.15.102:0/3865495964 conn(0x7ffbbc0fa9d0 msgr2=0x7ffbbc107e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:25.173 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:24.410+0000 7ffbc29bf640 1 -- 172.21.15.102:0/3865495964 shutdown_connections 2024-09-11T07:56:25.174 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:24.410+0000 7ffbc29bf640 1 -- 172.21.15.102:0/3865495964 wait complete. 2024-09-11T07:56:25.174 INFO:teuthology.orchestra.run.smithi102.stdout:mgr epoch 9 is available 2024-09-11T07:56:25.175 INFO:teuthology.orchestra.run.smithi102.stdout:Generating a dashboard self-signed certificate... 2024-09-11T07:56:25.420 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:25 smithi102 ceph-mon[29849]: mgrmap e11: y(active, since 1.00615s) 2024-09-11T07:56:25.421 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:25 smithi102 ceph-mon[29849]: [11/Sep/2024:07:56:24] ENGINE Bus STARTING 2024-09-11T07:56:25.421 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:25 smithi102 ceph-mon[29849]: [11/Sep/2024:07:56:24] ENGINE Serving on https://172.21.15.102:7150 2024-09-11T07:56:25.421 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:25 smithi102 ceph-mon[29849]: [11/Sep/2024:07:56:24] ENGINE Bus STARTED 2024-09-11T07:56:25.421 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:25 smithi102 ceph-mon[29849]: [11/Sep/2024:07:56:24] ENGINE Error in HTTPServer.serve 2024-09-11T07:56:25.421 INFO:journalctl@ceph.mon.a.smithi102.stdout: Traceback (most recent call last): 2024-09-11T07:56:25.421 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:56:25.421 INFO:journalctl@ceph.mon.a.smithi102.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:56:25.421 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:56:25.421 INFO:journalctl@ceph.mon.a.smithi102.stdout: self._run(expiration_interval) 2024-09-11T07:56:25.421 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:56:25.421 INFO:journalctl@ceph.mon.a.smithi102.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:56:25.422 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:56:25.422 INFO:journalctl@ceph.mon.a.smithi102.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:56:25.422 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:56:25.422 INFO:journalctl@ceph.mon.a.smithi102.stdout: s = self.context.wrap_socket( 2024-09-11T07:56:25.422 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:56:25.422 INFO:journalctl@ceph.mon.a.smithi102.stdout: return self.sslsocket_class._create( 2024-09-11T07:56:25.422 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:56:25.422 INFO:journalctl@ceph.mon.a.smithi102.stdout: self.do_handshake() 2024-09-11T07:56:25.422 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:56:25.422 INFO:journalctl@ceph.mon.a.smithi102.stdout: self._sslobj.do_handshake() 2024-09-11T07:56:25.422 INFO:journalctl@ceph.mon.a.smithi102.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:56:25.422 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:25 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:25.422 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:25 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:26.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-11T07:56:26.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.610+0000 7f28e8e61640 1 Processor -- start 2024-09-11T07:56:26.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.610+0000 7f28e8e61640 1 -- start start 2024-09-11T07:56:26.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.610+0000 7f28e8e61640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f28e41046a0 con 0x7f28e4073c30 2024-09-11T07:56:26.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.610+0000 7f28e259c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f28e4073c30 0x7f28e4102f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38520/0 (socket says 172.21.15.102:38520) 2024-09-11T07:56:26.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.610+0000 7f28e259c640 1 -- 172.21.15.102:0/1677857749 learned_addr learned my addr 172.21.15.102:0/1677857749 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:26.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.610+0000 7f28e1d9b640 1 -- 172.21.15.102:0/1677857749 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4266385955 0 0) 0x7f28e41046a0 con 0x7f28e4073c30 2024-09-11T07:56:26.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.610+0000 7f28e1d9b640 1 -- 172.21.15.102:0/1677857749 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28d00034e0 con 0x7f28e4073c30 2024-09-11T07:56:26.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.611+0000 7f28e1d9b640 1 -- 172.21.15.102:0/1677857749 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1510663682 0 0) 0x7f28d00034e0 con 0x7f28e4073c30 2024-09-11T07:56:26.466 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.611+0000 7f28e1d9b640 1 -- 172.21.15.102:0/1677857749 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e4105840 con 0x7f28e4073c30 2024-09-11T07:56:26.467 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.611+0000 7f28e1d9b640 1 -- 172.21.15.102:0/1677857749 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f28d4001910 con 0x7f28e4073c30 2024-09-11T07:56:26.467 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.611+0000 7f28e1d9b640 1 -- 172.21.15.102:0/1677857749 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f28d4001f80 con 0x7f28e4073c30 2024-09-11T07:56:26.467 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.611+0000 7f28e1d9b640 1 -- 172.21.15.102:0/1677857749 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f28d4005960 con 0x7f28e4073c30 2024-09-11T07:56:26.467 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.611+0000 7f28e8e61640 1 -- 172.21.15.102:0/1677857749 >> v1:172.21.15.102:6789/0 conn(0x7f28e4073c30 legacy=0x7f28e4102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:26.467 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.612+0000 7f28e8e61640 1 -- 172.21.15.102:0/1677857749 shutdown_connections 2024-09-11T07:56:26.467 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.612+0000 7f28e8e61640 1 -- 172.21.15.102:0/1677857749 >> 172.21.15.102:0/1677857749 conn(0x7f28e40fa9d0 msgr2=0x7f28e40fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:26.467 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.612+0000 7f28e8e61640 1 -- 172.21.15.102:0/1677857749 shutdown_connections 2024-09-11T07:56:26.467 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.612+0000 7f28e8e61640 1 -- 172.21.15.102:0/1677857749 wait complete. 2024-09-11T07:56:26.467 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.612+0000 7f28e8e61640 1 Processor -- start 2024-09-11T07:56:26.467 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.612+0000 7f28e8e61640 1 -- start start 2024-09-11T07:56:26.467 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.612+0000 7f28e8e61640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f28e41115e0 con 0x7f28e4073c30 2024-09-11T07:56:26.467 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.612+0000 7f28e259c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f28e4073c30 0x7f28e4110ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38536/0 (socket says 172.21.15.102:38536) 2024-09-11T07:56:26.468 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.612+0000 7f28e259c640 1 -- 172.21.15.102:0/1495718845 learned_addr learned my addr 172.21.15.102:0/1495718845 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:26.468 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.613+0000 7f28daffd640 1 -- 172.21.15.102:0/1495718845 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3393805314 0 0) 0x7f28e41115e0 con 0x7f28e4073c30 2024-09-11T07:56:26.468 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.613+0000 7f28daffd640 1 -- 172.21.15.102:0/1495718845 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28d0005440 con 0x7f28e4073c30 2024-09-11T07:56:26.468 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.613+0000 7f28daffd640 1 -- 172.21.15.102:0/1495718845 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1710732073 0 0) 0x7f28d0005440 con 0x7f28e4073c30 2024-09-11T07:56:26.468 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.613+0000 7f28daffd640 1 -- 172.21.15.102:0/1495718845 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f28e41115e0 con 0x7f28e4073c30 2024-09-11T07:56:26.468 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.613+0000 7f28daffd640 1 -- 172.21.15.102:0/1495718845 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f28d4004c40 con 0x7f28e4073c30 2024-09-11T07:56:26.468 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.613+0000 7f28daffd640 1 -- 172.21.15.102:0/1495718845 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2341908839 0 0) 0x7f28e41115e0 con 0x7f28e4073c30 2024-09-11T07:56:26.468 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.613+0000 7f28daffd640 1 -- 172.21.15.102:0/1495718845 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e4111770 con 0x7f28e4073c30 2024-09-11T07:56:26.468 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.613+0000 7f28e8e61640 1 -- 172.21.15.102:0/1495718845 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e4111a40 con 0x7f28e4073c30 2024-09-11T07:56:26.468 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.613+0000 7f28e8e61640 1 -- 172.21.15.102:0/1495718845 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e41154d0 con 0x7f28e4073c30 2024-09-11T07:56:26.468 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.613+0000 7f28daffd640 1 -- 172.21.15.102:0/1495718845 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f28d4001b80 con 0x7f28e4073c30 2024-09-11T07:56:26.468 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.613+0000 7f28daffd640 1 -- 172.21.15.102:0/1495718845 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f28d4006460 con 0x7f28e4073c30 2024-09-11T07:56:26.468 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.614+0000 7f28daffd640 1 -- 172.21.15.102:0/1495718845 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 11) v1 ==== 45364+0+0 (unknown 1346339249 0 0) 0x7f28d4011700 con 0x7f28e4073c30 2024-09-11T07:56:26.469 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.614+0000 7f28e8e61640 1 -- 172.21.15.102:0/1495718845 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c4005040 con 0x7f28e4073c30 2024-09-11T07:56:26.469 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.614+0000 7f28daffd640 1 -- 172.21.15.102:0/1495718845 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (unknown 4267545286 0 0) 0x7f28d4047d00 con 0x7f28e4073c30 2024-09-11T07:56:26.469 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.619+0000 7f28daffd640 1 -- 172.21.15.102:0/1495718845 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f28d4012f30 con 0x7f28e4073c30 2024-09-11T07:56:26.469 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.785+0000 7f28e8e61640 1 -- 172.21.15.102:0/1495718845 --> v1:172.21.15.102:6800/2950771867 -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f28c4000bc0 con 0x7f28d003a5d0 2024-09-11T07:56:26.469 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.879+0000 7f28daffd640 1 -- 172.21.15.102:0/1495718845 <== mgr.14150 v1:172.21.15.102:6800/2950771867 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (unknown 0 0 3317752739) 0x7f28c4000bc0 con 0x7f28d003a5d0 2024-09-11T07:56:26.469 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.881+0000 7f28e8e61640 1 -- 172.21.15.102:0/1495718845 >> v1:172.21.15.102:6800/2950771867 conn(0x7f28d003a5d0 legacy=0x7f28d003ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:26.469 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.881+0000 7f28e8e61640 1 -- 172.21.15.102:0/1495718845 >> v1:172.21.15.102:6789/0 conn(0x7f28e4073c30 legacy=0x7f28e4110ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:26.469 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.881+0000 7f28e8e61640 1 -- 172.21.15.102:0/1495718845 shutdown_connections 2024-09-11T07:56:26.469 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.881+0000 7f28e8e61640 1 -- 172.21.15.102:0/1495718845 >> 172.21.15.102:0/1495718845 conn(0x7f28e40fa9d0 msgr2=0x7f28e40fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:26.470 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.881+0000 7f28e8e61640 1 -- 172.21.15.102:0/1495718845 shutdown_connections 2024-09-11T07:56:26.470 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:25.881+0000 7f28e8e61640 1 -- 172.21.15.102:0/1495718845 wait complete. 2024-09-11T07:56:26.470 INFO:teuthology.orchestra.run.smithi102.stdout:Creating initial admin user... 2024-09-11T07:56:27.046 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:26 smithi102 ceph-mon[29849]: from='client.14162 v1:172.21.15.102:0/1495718845' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:56:27.047 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:27.047 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:27.047 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:27.047 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:26 smithi102 ceph-mon[29849]: mgrmap e12: y(active, since 2s) 2024-09-11T07:56:28.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$onE2Y49mq072TWR9X9nDCugEfP.VfMDNaEmOPAZFvAkBidt4SlLIS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726041387, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-11T07:56:28.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.856+0000 7fb080b9e640 1 Processor -- start 2024-09-11T07:56:28.085 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.856+0000 7fb080b9e640 1 -- start start 2024-09-11T07:56:28.086 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.856+0000 7fb080b9e640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb07c104590 con 0x7fb07c074460 2024-09-11T07:56:28.086 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.857+0000 7fb07b7fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb07c074460 0x7fb07c102e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38540/0 (socket says 172.21.15.102:38540) 2024-09-11T07:56:28.086 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.857+0000 7fb07b7fe640 1 -- 172.21.15.102:0/3734299702 learned_addr learned my addr 172.21.15.102:0/3734299702 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:28.086 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.857+0000 7fb07affd640 1 -- 172.21.15.102:0/3734299702 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4001001309 0 0) 0x7fb07c104590 con 0x7fb07c074460 2024-09-11T07:56:28.086 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.857+0000 7fb07affd640 1 -- 172.21.15.102:0/3734299702 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0680034e0 con 0x7fb07c074460 2024-09-11T07:56:28.086 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.857+0000 7fb07affd640 1 -- 172.21.15.102:0/3734299702 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1810186576 0 0) 0x7fb0680034e0 con 0x7fb07c074460 2024-09-11T07:56:28.086 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.857+0000 7fb07affd640 1 -- 172.21.15.102:0/3734299702 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07c105730 con 0x7fb07c074460 2024-09-11T07:56:28.086 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.857+0000 7fb07affd640 1 -- 172.21.15.102:0/3734299702 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fb06c001590 con 0x7fb07c074460 2024-09-11T07:56:28.086 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.857+0000 7fb07affd640 1 -- 172.21.15.102:0/3734299702 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7fb06c001c00 con 0x7fb07c074460 2024-09-11T07:56:28.086 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.857+0000 7fb07affd640 1 -- 172.21.15.102:0/3734299702 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fb06c0055e0 con 0x7fb07c074460 2024-09-11T07:56:28.086 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.858+0000 7fb080b9e640 1 -- 172.21.15.102:0/3734299702 >> v1:172.21.15.102:6789/0 conn(0x7fb07c074460 legacy=0x7fb07c102e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:28.087 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.858+0000 7fb080b9e640 1 -- 172.21.15.102:0/3734299702 shutdown_connections 2024-09-11T07:56:28.087 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.858+0000 7fb080b9e640 1 -- 172.21.15.102:0/3734299702 >> 172.21.15.102:0/3734299702 conn(0x7fb07c0fa900 msgr2=0x7fb07c0fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:28.087 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.858+0000 7fb080b9e640 1 -- 172.21.15.102:0/3734299702 shutdown_connections 2024-09-11T07:56:28.087 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.858+0000 7fb080b9e640 1 -- 172.21.15.102:0/3734299702 wait complete. 2024-09-11T07:56:28.087 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.858+0000 7fb080b9e640 1 Processor -- start 2024-09-11T07:56:28.087 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.858+0000 7fb080b9e640 1 -- start start 2024-09-11T07:56:28.087 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.858+0000 7fb080b9e640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb07c10f320 con 0x7fb07c074460 2024-09-11T07:56:28.087 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.859+0000 7fb07b7fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb07c074460 0x7fb07c10ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:38552/0 (socket says 172.21.15.102:38552) 2024-09-11T07:56:28.087 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.859+0000 7fb07b7fe640 1 -- 172.21.15.102:0/2557369607 learned_addr learned my addr 172.21.15.102:0/2557369607 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:28.087 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.859+0000 7fb078ff9640 1 -- 172.21.15.102:0/2557369607 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1215735775 0 0) 0x7fb07c10f320 con 0x7fb07c074460 2024-09-11T07:56:28.087 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.859+0000 7fb078ff9640 1 -- 172.21.15.102:0/2557369607 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb068005440 con 0x7fb07c074460 2024-09-11T07:56:28.088 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.859+0000 7fb078ff9640 1 -- 172.21.15.102:0/2557369607 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 515929359 0 0) 0x7fb068005440 con 0x7fb07c074460 2024-09-11T07:56:28.088 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.859+0000 7fb078ff9640 1 -- 172.21.15.102:0/2557369607 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb07c10f320 con 0x7fb07c074460 2024-09-11T07:56:28.088 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.859+0000 7fb078ff9640 1 -- 172.21.15.102:0/2557369607 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fb06c0048c0 con 0x7fb07c074460 2024-09-11T07:56:28.088 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.859+0000 7fb078ff9640 1 -- 172.21.15.102:0/2557369607 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1194174555 0 0) 0x7fb07c10f320 con 0x7fb07c074460 2024-09-11T07:56:28.088 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.859+0000 7fb078ff9640 1 -- 172.21.15.102:0/2557369607 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07c10f4b0 con 0x7fb07c074460 2024-09-11T07:56:28.088 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.859+0000 7fb080b9e640 1 -- 172.21.15.102:0/2557369607 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07c10f780 con 0x7fb07c074460 2024-09-11T07:56:28.088 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.859+0000 7fb080b9e640 1 -- 172.21.15.102:0/2557369607 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07c113340 con 0x7fb07c074460 2024-09-11T07:56:28.088 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.859+0000 7fb078ff9640 1 -- 172.21.15.102:0/2557369607 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7fb06c001800 con 0x7fb07c074460 2024-09-11T07:56:28.088 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.859+0000 7fb078ff9640 1 -- 172.21.15.102:0/2557369607 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fb06c006140 con 0x7fb07c074460 2024-09-11T07:56:28.088 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.860+0000 7fb078ff9640 1 -- 172.21.15.102:0/2557369607 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 12) v1 ==== 45470+0+0 (unknown 2467458955 0 0) 0x7fb06c011450 con 0x7fb07c074460 2024-09-11T07:56:28.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.860+0000 7fb078ff9640 1 -- 172.21.15.102:0/2557369607 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (unknown 4267545286 0 0) 0x7fb06c047ac0 con 0x7fb07c074460 2024-09-11T07:56:28.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.860+0000 7fb080b9e640 1 -- 172.21.15.102:0/2557369607 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb05c005040 con 0x7fb07c074460 2024-09-11T07:56:28.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:26.865+0000 7fb078ff9640 1 -- 172.21.15.102:0/2557369607 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fb06c016b60 con 0x7fb07c074460 2024-09-11T07:56:28.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:27.045+0000 7fb080b9e640 1 -- 172.21.15.102:0/2557369607 --> v1:172.21.15.102:6800/2950771867 -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fb05c002420 con 0x7fb06803a7c0 2024-09-11T07:56:28.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:27.271+0000 7fb078ff9640 1 -- 172.21.15.102:0/2557369607 <== mgr.14150 v1:172.21.15.102:6800/2950771867 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (unknown 0 0 2269220476) 0x7fb05c002420 con 0x7fb06803a7c0 2024-09-11T07:56:28.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:27.273+0000 7fb080b9e640 1 -- 172.21.15.102:0/2557369607 >> v1:172.21.15.102:6800/2950771867 conn(0x7fb06803a7c0 legacy=0x7fb06803cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:28.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:27.273+0000 7fb080b9e640 1 -- 172.21.15.102:0/2557369607 >> v1:172.21.15.102:6789/0 conn(0x7fb07c074460 legacy=0x7fb07c10ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:28.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:27.274+0000 7fb080b9e640 1 -- 172.21.15.102:0/2557369607 shutdown_connections 2024-09-11T07:56:28.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:27.274+0000 7fb080b9e640 1 -- 172.21.15.102:0/2557369607 >> 172.21.15.102:0/2557369607 conn(0x7fb07c0fa900 msgr2=0x7fb07c105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:28.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:27.274+0000 7fb080b9e640 1 -- 172.21.15.102:0/2557369607 shutdown_connections 2024-09-11T07:56:28.089 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:27.274+0000 7fb080b9e640 1 -- 172.21.15.102:0/2557369607 wait complete. 2024-09-11T07:56:28.090 INFO:teuthology.orchestra.run.smithi102.stdout:Fetching dashboard port number... 2024-09-11T07:56:28.500 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:28 smithi102 ceph-mon[29849]: from='client.14164 v1:172.21.15.102:0/2557369607' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:56:28.500 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:28 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:29.520 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2532082428' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-11T07:56:29.527 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stdout 8443 2024-09-11T07:56:29.527 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.498+0000 7f9600857640 1 Processor -- start 2024-09-11T07:56:29.527 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.499+0000 7f9600857640 1 -- start start 2024-09-11T07:56:29.527 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.499+0000 7f9600857640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f95fc104740 con 0x7f95fc073b90 2024-09-11T07:56:29.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.499+0000 7f95fa59c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f95fc073b90 0x7f95fc103020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:42822/0 (socket says 172.21.15.102:42822) 2024-09-11T07:56:29.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.499+0000 7f95fa59c640 1 -- 172.21.15.102:0/4216577514 learned_addr learned my addr 172.21.15.102:0/4216577514 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:29.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.499+0000 7f95f959a640 1 -- 172.21.15.102:0/4216577514 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3342100270 0 0) 0x7f95fc104740 con 0x7f95fc073b90 2024-09-11T07:56:29.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.499+0000 7f95f959a640 1 -- 172.21.15.102:0/4216577514 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95e40034e0 con 0x7f95fc073b90 2024-09-11T07:56:29.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.500+0000 7f95f959a640 1 -- 172.21.15.102:0/4216577514 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2865375483 0 0) 0x7f95e40034e0 con 0x7f95fc073b90 2024-09-11T07:56:29.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.500+0000 7f95f959a640 1 -- 172.21.15.102:0/4216577514 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc1058e0 con 0x7f95fc073b90 2024-09-11T07:56:29.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.500+0000 7f95f959a640 1 -- 172.21.15.102:0/4216577514 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f95ec001590 con 0x7f95fc073b90 2024-09-11T07:56:29.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.500+0000 7f95f959a640 1 -- 172.21.15.102:0/4216577514 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f95ec001c00 con 0x7f95fc073b90 2024-09-11T07:56:29.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.500+0000 7f95f959a640 1 -- 172.21.15.102:0/4216577514 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f95ec0055e0 con 0x7f95fc073b90 2024-09-11T07:56:29.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.501+0000 7f9600857640 1 -- 172.21.15.102:0/4216577514 >> v1:172.21.15.102:6789/0 conn(0x7f95fc073b90 legacy=0x7f95fc103020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:29.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.501+0000 7f9600857640 1 -- 172.21.15.102:0/4216577514 shutdown_connections 2024-09-11T07:56:29.528 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.501+0000 7f9600857640 1 -- 172.21.15.102:0/4216577514 >> 172.21.15.102:0/4216577514 conn(0x7f95fc0fa9d0 msgr2=0x7f95fc0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:29.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.501+0000 7f9600857640 1 -- 172.21.15.102:0/4216577514 shutdown_connections 2024-09-11T07:56:29.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.501+0000 7f9600857640 1 -- 172.21.15.102:0/4216577514 wait complete. 2024-09-11T07:56:29.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.501+0000 7f9600857640 1 Processor -- start 2024-09-11T07:56:29.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.502+0000 7f9600857640 1 -- start start 2024-09-11T07:56:29.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.502+0000 7f9600857640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f95fc10f420 con 0x7f95fc073b90 2024-09-11T07:56:29.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.502+0000 7f95fa59c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f95fc073b90 0x7f95fc10ed10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:42828/0 (socket says 172.21.15.102:42828) 2024-09-11T07:56:29.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.502+0000 7f95fa59c640 1 -- 172.21.15.102:0/2532082428 learned_addr learned my addr 172.21.15.102:0/2532082428 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:29.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.502+0000 7f95eaffd640 1 -- 172.21.15.102:0/2532082428 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3634556243 0 0) 0x7f95fc10f420 con 0x7f95fc073b90 2024-09-11T07:56:29.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.502+0000 7f95eaffd640 1 -- 172.21.15.102:0/2532082428 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95e4005440 con 0x7f95fc073b90 2024-09-11T07:56:29.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.502+0000 7f95eaffd640 1 -- 172.21.15.102:0/2532082428 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1308104801 0 0) 0x7f95e4005440 con 0x7f95fc073b90 2024-09-11T07:56:29.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.502+0000 7f95eaffd640 1 -- 172.21.15.102:0/2532082428 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f95fc10f420 con 0x7f95fc073b90 2024-09-11T07:56:29.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.502+0000 7f95eaffd640 1 -- 172.21.15.102:0/2532082428 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f95ec0048c0 con 0x7f95fc073b90 2024-09-11T07:56:29.529 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.502+0000 7f95eaffd640 1 -- 172.21.15.102:0/2532082428 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2402876305 0 0) 0x7f95fc10f420 con 0x7f95fc073b90 2024-09-11T07:56:29.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.502+0000 7f95eaffd640 1 -- 172.21.15.102:0/2532082428 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc10f5b0 con 0x7f95fc073b90 2024-09-11T07:56:29.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.502+0000 7f9600857640 1 -- 172.21.15.102:0/2532082428 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95fc10f880 con 0x7f95fc073b90 2024-09-11T07:56:29.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.502+0000 7f9600857640 1 -- 172.21.15.102:0/2532082428 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f95fc113310 con 0x7f95fc073b90 2024-09-11T07:56:29.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.502+0000 7f95eaffd640 1 -- 172.21.15.102:0/2532082428 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f95ec001800 con 0x7f95fc073b90 2024-09-11T07:56:29.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.503+0000 7f95eaffd640 1 -- 172.21.15.102:0/2532082428 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f95ec006060 con 0x7f95fc073b90 2024-09-11T07:56:29.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.503+0000 7f95eaffd640 1 -- 172.21.15.102:0/2532082428 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 12) v1 ==== 45470+0+0 (unknown 2467458955 0 0) 0x7f95ec011370 con 0x7f95fc073b90 2024-09-11T07:56:29.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.503+0000 7f9600857640 1 -- 172.21.15.102:0/2532082428 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95dc005040 con 0x7f95fc073b90 2024-09-11T07:56:29.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.503+0000 7f95eaffd640 1 -- 172.21.15.102:0/2532082428 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (unknown 4267545286 0 0) 0x7f95ec048810 con 0x7f95fc073b90 2024-09-11T07:56:29.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.508+0000 7f95eaffd640 1 -- 172.21.15.102:0/2532082428 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f95ec012b50 con 0x7f95fc073b90 2024-09-11T07:56:29.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.674+0000 7f9600857640 1 -- 172.21.15.102:0/2532082428 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f95dc005870 con 0x7f95fc073b90 2024-09-11T07:56:29.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.675+0000 7f95eaffd640 1 -- 172.21.15.102:0/2532082428 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (unknown 3713421687 0 83753974) 0x7f95ec0165a0 con 0x7f95fc073b90 2024-09-11T07:56:29.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.677+0000 7f9600857640 1 -- 172.21.15.102:0/2532082428 >> v1:172.21.15.102:6800/2950771867 conn(0x7f95e403a7c0 legacy=0x7f95e403cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:29.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.677+0000 7f9600857640 1 -- 172.21.15.102:0/2532082428 >> v1:172.21.15.102:6789/0 conn(0x7f95fc073b90 legacy=0x7f95fc10ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:29.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.677+0000 7f9600857640 1 -- 172.21.15.102:0/2532082428 shutdown_connections 2024-09-11T07:56:29.530 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.677+0000 7f9600857640 1 -- 172.21.15.102:0/2532082428 >> 172.21.15.102:0/2532082428 conn(0x7f95fc0fa9d0 msgr2=0x7f95fc102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:29.531 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.677+0000 7f9600857640 1 -- 172.21.15.102:0/2532082428 shutdown_connections 2024-09-11T07:56:29.531 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:28.677+0000 7f9600857640 1 -- 172.21.15.102:0/2532082428 wait complete. 2024-09-11T07:56:29.534 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T07:56:29.534 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout disabled 2024-09-11T07:56:29.541 INFO:teuthology.orchestra.run.smithi102.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T07:56:29.541 INFO:teuthology.orchestra.run.smithi102.stdout:systemctl: stdout inactive 2024-09-11T07:56:29.541 INFO:teuthology.orchestra.run.smithi102.stdout:firewalld.service is not enabled 2024-09-11T07:56:29.542 INFO:teuthology.orchestra.run.smithi102.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-11T07:56:29.543 INFO:teuthology.orchestra.run.smithi102.stdout:Ceph Dashboard is now available at: 2024-09-11T07:56:29.544 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:56:29.544 INFO:teuthology.orchestra.run.smithi102.stdout: URL: https://smithi102.front.sepia.ceph.com:8443/ 2024-09-11T07:56:29.544 INFO:teuthology.orchestra.run.smithi102.stdout: User: admin 2024-09-11T07:56:29.544 INFO:teuthology.orchestra.run.smithi102.stdout: Password: gdw34vh5ys 2024-09-11T07:56:29.544 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:56:29.544 INFO:teuthology.orchestra.run.smithi102.stdout:Saving cluster configuration to /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config directory 2024-09-11T07:56:29.544 INFO:teuthology.orchestra.run.smithi102.stdout:Enabling autotune for osd_memory_target 2024-09-11T07:56:30.866 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.076+0000 7f48b46da640 1 Processor -- start 2024-09-11T07:56:30.866 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.076+0000 7f48b46da640 1 -- start start 2024-09-11T07:56:30.866 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.076+0000 7f48b46da640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f48ac1046d0 con 0x7f48ac0745b0 2024-09-11T07:56:30.866 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.076+0000 7f48b2476640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f48ac0745b0 0x7f48ac102fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:42842/0 (socket says 172.21.15.102:42842) 2024-09-11T07:56:30.866 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.076+0000 7f48b2476640 1 -- 172.21.15.102:0/3331838456 learned_addr learned my addr 172.21.15.102:0/3331838456 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:30.867 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.076+0000 7f48b1474640 1 -- 172.21.15.102:0/3331838456 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2247216101 0 0) 0x7f48ac1046d0 con 0x7f48ac0745b0 2024-09-11T07:56:30.867 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.076+0000 7f48b1474640 1 -- 172.21.15.102:0/3331838456 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48980034e0 con 0x7f48ac0745b0 2024-09-11T07:56:30.867 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.076+0000 7f48b1474640 1 -- 172.21.15.102:0/3331838456 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4221852599 0 0) 0x7f48980034e0 con 0x7f48ac0745b0 2024-09-11T07:56:30.867 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.077+0000 7f48b1474640 1 -- 172.21.15.102:0/3331838456 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ac105870 con 0x7f48ac0745b0 2024-09-11T07:56:30.867 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.077+0000 7f48b1474640 1 -- 172.21.15.102:0/3331838456 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f48a8001590 con 0x7f48ac0745b0 2024-09-11T07:56:30.867 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.077+0000 7f48b1474640 1 -- 172.21.15.102:0/3331838456 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f48a8001cd0 con 0x7f48ac0745b0 2024-09-11T07:56:30.867 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.077+0000 7f48b1474640 1 -- 172.21.15.102:0/3331838456 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f48a80056c0 con 0x7f48ac0745b0 2024-09-11T07:56:30.867 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.077+0000 7f48b46da640 1 -- 172.21.15.102:0/3331838456 >> v1:172.21.15.102:6789/0 conn(0x7f48ac0745b0 legacy=0x7f48ac102fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:30.867 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.077+0000 7f48b46da640 1 -- 172.21.15.102:0/3331838456 shutdown_connections 2024-09-11T07:56:30.867 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.077+0000 7f48b46da640 1 -- 172.21.15.102:0/3331838456 >> 172.21.15.102:0/3331838456 conn(0x7f48ac0faaa0 msgr2=0x7f48ac0fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:30.867 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.077+0000 7f48b46da640 1 -- 172.21.15.102:0/3331838456 shutdown_connections 2024-09-11T07:56:30.867 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.077+0000 7f48b46da640 1 -- 172.21.15.102:0/3331838456 wait complete. 2024-09-11T07:56:30.867 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.078+0000 7f48b46da640 1 Processor -- start 2024-09-11T07:56:30.868 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.078+0000 7f48b46da640 1 -- start start 2024-09-11T07:56:30.868 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.078+0000 7f48b46da640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f48ac10f3c0 con 0x7f48ac0745b0 2024-09-11T07:56:30.868 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.078+0000 7f48b2476640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f48ac0745b0 0x7f48ac10ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:42846/0 (socket says 172.21.15.102:42846) 2024-09-11T07:56:30.868 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.078+0000 7f48b2476640 1 -- 172.21.15.102:0/4252915642 learned_addr learned my addr 172.21.15.102:0/4252915642 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:30.868 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.078+0000 7f48a2ffd640 1 -- 172.21.15.102:0/4252915642 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3036871707 0 0) 0x7f48ac10f3c0 con 0x7f48ac0745b0 2024-09-11T07:56:30.868 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.078+0000 7f48a2ffd640 1 -- 172.21.15.102:0/4252915642 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48980053f0 con 0x7f48ac0745b0 2024-09-11T07:56:30.868 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.078+0000 7f48a2ffd640 1 -- 172.21.15.102:0/4252915642 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 400673182 0 0) 0x7f48980053f0 con 0x7f48ac0745b0 2024-09-11T07:56:30.868 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.078+0000 7f48a2ffd640 1 -- 172.21.15.102:0/4252915642 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f48ac10f3c0 con 0x7f48ac0745b0 2024-09-11T07:56:30.869 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.078+0000 7f48a2ffd640 1 -- 172.21.15.102:0/4252915642 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f48a8001430 con 0x7f48ac0745b0 2024-09-11T07:56:30.869 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.078+0000 7f48a2ffd640 1 -- 172.21.15.102:0/4252915642 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3760509590 0 0) 0x7f48ac10f3c0 con 0x7f48ac0745b0 2024-09-11T07:56:30.869 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.079+0000 7f48a2ffd640 1 -- 172.21.15.102:0/4252915642 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ac10f550 con 0x7f48ac0745b0 2024-09-11T07:56:30.869 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.079+0000 7f48b46da640 1 -- 172.21.15.102:0/4252915642 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ac10f820 con 0x7f48ac0745b0 2024-09-11T07:56:30.869 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.079+0000 7f48b46da640 1 -- 172.21.15.102:0/4252915642 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ac1132b0 con 0x7f48ac0745b0 2024-09-11T07:56:30.869 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.079+0000 7f48a2ffd640 1 -- 172.21.15.102:0/4252915642 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f48a8001b30 con 0x7f48ac0745b0 2024-09-11T07:56:30.869 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.079+0000 7f48a2ffd640 1 -- 172.21.15.102:0/4252915642 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f48a8006150 con 0x7f48ac0745b0 2024-09-11T07:56:30.869 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.079+0000 7f48a2ffd640 1 -- 172.21.15.102:0/4252915642 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 12) v1 ==== 45470+0+0 (unknown 2467458955 0 0) 0x7f48a8011520 con 0x7f48ac0745b0 2024-09-11T07:56:30.869 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.079+0000 7f48b46da640 1 -- 172.21.15.102:0/4252915642 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4890005040 con 0x7f48ac0745b0 2024-09-11T07:56:30.869 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.080+0000 7f48a2ffd640 1 -- 172.21.15.102:0/4252915642 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (unknown 4267545286 0 0) 0x7f48a8047c10 con 0x7f48ac0745b0 2024-09-11T07:56:30.869 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.084+0000 7f48a2ffd640 1 -- 172.21.15.102:0/4252915642 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f48a8012db0 con 0x7f48ac0745b0 2024-09-11T07:56:30.869 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.240+0000 7f48b46da640 1 -- 172.21.15.102:0/4252915642 --> v1:172.21.15.102:6789/0 -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f4890005870 con 0x7f48ac0745b0 2024-09-11T07:56:30.869 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.240+0000 7f48a2ffd640 1 -- 172.21.15.102:0/4252915642 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (unknown 689008156 0 0) 0x7f48a8016800 con 0x7f48ac0745b0 2024-09-11T07:56:30.870 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.242+0000 7f48b46da640 1 -- 172.21.15.102:0/4252915642 >> v1:172.21.15.102:6800/2950771867 conn(0x7f489803a570 legacy=0x7f489803ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:30.870 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.242+0000 7f48b46da640 1 -- 172.21.15.102:0/4252915642 >> v1:172.21.15.102:6789/0 conn(0x7f48ac0745b0 legacy=0x7f48ac10ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:30.870 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.242+0000 7f48b46da640 1 -- 172.21.15.102:0/4252915642 shutdown_connections 2024-09-11T07:56:30.870 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.242+0000 7f48b46da640 1 -- 172.21.15.102:0/4252915642 >> 172.21.15.102:0/4252915642 conn(0x7f48ac0faaa0 msgr2=0x7f48ac105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:30.870 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.242+0000 7f48b46da640 1 -- 172.21.15.102:0/4252915642 shutdown_connections 2024-09-11T07:56:30.870 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:30.242+0000 7f48b46da640 1 -- 172.21.15.102:0/4252915642 wait complete. 2024-09-11T07:56:31.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:31 smithi102 ceph-mon[29849]: mgrmap e13: y(active, since 6s) 2024-09-11T07:56:32.156 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.323+0000 7f6ee237e640 1 Processor -- start 2024-09-11T07:56:32.156 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.323+0000 7f6ee237e640 1 -- start start 2024-09-11T07:56:32.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.323+0000 7f6ee237e640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6edc0ffd70 con 0x7f6edc0ff250 2024-09-11T07:56:32.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.324+0000 7f6edbfff640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f6edc0ff250 0x7f6edc0ff660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:42862/0 (socket says 172.21.15.102:42862) 2024-09-11T07:56:32.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.324+0000 7f6edbfff640 1 -- 172.21.15.102:0/2928746053 learned_addr learned my addr 172.21.15.102:0/2928746053 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:32.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.324+0000 7f6edaffd640 1 -- 172.21.15.102:0/2928746053 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3658417138 0 0) 0x7f6edc0ffd70 con 0x7f6edc0ff250 2024-09-11T07:56:32.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.324+0000 7f6edaffd640 1 -- 172.21.15.102:0/2928746053 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ec40034e0 con 0x7f6edc0ff250 2024-09-11T07:56:32.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.324+0000 7f6edaffd640 1 -- 172.21.15.102:0/2928746053 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4105721328 0 0) 0x7f6ec40034e0 con 0x7f6edc0ff250 2024-09-11T07:56:32.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.324+0000 7f6edaffd640 1 -- 172.21.15.102:0/2928746053 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc104710 con 0x7f6edc0ff250 2024-09-11T07:56:32.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.324+0000 7f6edaffd640 1 -- 172.21.15.102:0/2928746053 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f6ecc001590 con 0x7f6edc0ff250 2024-09-11T07:56:32.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.324+0000 7f6edaffd640 1 -- 172.21.15.102:0/2928746053 <== mon.0 v1:172.21.15.102:6789/0 4 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f6ecc001cd0 con 0x7f6edc0ff250 2024-09-11T07:56:32.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.324+0000 7f6edaffd640 1 -- 172.21.15.102:0/2928746053 <== mon.0 v1:172.21.15.102:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f6ecc0056c0 con 0x7f6edc0ff250 2024-09-11T07:56:32.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.325+0000 7f6ee237e640 1 -- 172.21.15.102:0/2928746053 >> v1:172.21.15.102:6789/0 conn(0x7f6edc0ff250 legacy=0x7f6edc0ff660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:32.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.325+0000 7f6ee237e640 1 -- 172.21.15.102:0/2928746053 shutdown_connections 2024-09-11T07:56:32.157 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.325+0000 7f6ee237e640 1 -- 172.21.15.102:0/2928746053 >> 172.21.15.102:0/2928746053 conn(0x7f6edc0fa9d0 msgr2=0x7f6edc0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:32.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.325+0000 7f6ee237e640 1 -- 172.21.15.102:0/2928746053 shutdown_connections 2024-09-11T07:56:32.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.325+0000 7f6ee237e640 1 -- 172.21.15.102:0/2928746053 wait complete. 2024-09-11T07:56:32.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.325+0000 7f6ee237e640 1 Processor -- start 2024-09-11T07:56:32.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.325+0000 7f6ee237e640 1 -- start start 2024-09-11T07:56:32.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.325+0000 7f6ee237e640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6edc10f380 con 0x7f6edc0ff250 2024-09-11T07:56:32.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.326+0000 7f6edbfff640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f6edc0ff250 0x7f6edc10ec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:42866/0 (socket says 172.21.15.102:42866) 2024-09-11T07:56:32.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.326+0000 7f6edbfff640 1 -- 172.21.15.102:0/4208325541 learned_addr learned my addr 172.21.15.102:0/4208325541 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:32.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.326+0000 7f6ed8ff9640 1 -- 172.21.15.102:0/4208325541 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954316063 0 0) 0x7f6edc10f380 con 0x7f6edc0ff250 2024-09-11T07:56:32.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.326+0000 7f6ed8ff9640 1 -- 172.21.15.102:0/4208325541 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ec40053f0 con 0x7f6edc0ff250 2024-09-11T07:56:32.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.326+0000 7f6ed8ff9640 1 -- 172.21.15.102:0/4208325541 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 328624712 0 0) 0x7f6ec40053f0 con 0x7f6edc0ff250 2024-09-11T07:56:32.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.326+0000 7f6ed8ff9640 1 -- 172.21.15.102:0/4208325541 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6edc10f380 con 0x7f6edc0ff250 2024-09-11T07:56:32.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.326+0000 7f6ed8ff9640 1 -- 172.21.15.102:0/4208325541 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f6ecc001430 con 0x7f6edc0ff250 2024-09-11T07:56:32.158 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.326+0000 7f6ed8ff9640 1 -- 172.21.15.102:0/4208325541 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 234116346 0 0) 0x7f6edc10f380 con 0x7f6edc0ff250 2024-09-11T07:56:32.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.326+0000 7f6ed8ff9640 1 -- 172.21.15.102:0/4208325541 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc10f510 con 0x7f6edc0ff250 2024-09-11T07:56:32.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.326+0000 7f6ee237e640 1 -- 172.21.15.102:0/4208325541 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6edc10f7e0 con 0x7f6edc0ff250 2024-09-11T07:56:32.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.326+0000 7f6ee237e640 1 -- 172.21.15.102:0/4208325541 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6edc113270 con 0x7f6edc0ff250 2024-09-11T07:56:32.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.326+0000 7f6ed8ff9640 1 -- 172.21.15.102:0/4208325541 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f6ecc001b30 con 0x7f6edc0ff250 2024-09-11T07:56:32.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.326+0000 7f6ed8ff9640 1 -- 172.21.15.102:0/4208325541 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f6ecc0061c0 con 0x7f6edc0ff250 2024-09-11T07:56:32.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.327+0000 7f6ed8ff9640 1 -- 172.21.15.102:0/4208325541 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 1600948874 0 0) 0x7f6ecc011500 con 0x7f6edc0ff250 2024-09-11T07:56:32.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.327+0000 7f6ee237e640 1 -- 172.21.15.102:0/4208325541 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec8005040 con 0x7f6edc0ff250 2024-09-11T07:56:32.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.327+0000 7f6ed8ff9640 1 -- 172.21.15.102:0/4208325541 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (unknown 4267545286 0 0) 0x7f6ecc047bb0 con 0x7f6edc0ff250 2024-09-11T07:56:32.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.332+0000 7f6ed8ff9640 1 -- 172.21.15.102:0/4208325541 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f6ecc016d80 con 0x7f6edc0ff250 2024-09-11T07:56:32.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.568+0000 7f6ee237e640 1 -- 172.21.15.102:0/4208325541 --> v1:172.21.15.102:6789/0 -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f6ec8005870 con 0x7f6edc0ff250 2024-09-11T07:56:32.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.571+0000 7f6ed8ff9640 1 -- 172.21.15.102:0/4208325541 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (unknown 1169358022 0 0) 0x7f6ecc016780 con 0x7f6edc0ff250 2024-09-11T07:56:32.159 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-11T07:56:32.160 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.573+0000 7f6ee237e640 1 -- 172.21.15.102:0/4208325541 >> v1:172.21.15.102:6800/2950771867 conn(0x7f6ec403a710 legacy=0x7f6ec403cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:32.160 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.573+0000 7f6ee237e640 1 -- 172.21.15.102:0/4208325541 >> v1:172.21.15.102:6789/0 conn(0x7f6edc0ff250 legacy=0x7f6edc10ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:32.160 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.573+0000 7f6ee237e640 1 -- 172.21.15.102:0/4208325541 shutdown_connections 2024-09-11T07:56:32.160 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.573+0000 7f6ee237e640 1 -- 172.21.15.102:0/4208325541 >> 172.21.15.102:0/4208325541 conn(0x7f6edc0fa9d0 msgr2=0x7f6edc072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:32.160 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.573+0000 7f6ee237e640 1 -- 172.21.15.102:0/4208325541 shutdown_connections 2024-09-11T07:56:32.160 INFO:teuthology.orchestra.run.smithi102.stdout:/usr/bin/ceph: stderr 2024-09-11T07:56:31.573+0000 7f6ee237e640 1 -- 172.21.15.102:0/4208325541 wait complete. 2024-09-11T07:56:32.160 INFO:teuthology.orchestra.run.smithi102.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-11T07:56:32.160 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:56:32.160 INFO:teuthology.orchestra.run.smithi102.stdout: sudo /sbin/cephadm shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:56:32.160 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:56:32.160 INFO:teuthology.orchestra.run.smithi102.stdout:Or, if you are only running a single cluster on this host: 2024-09-11T07:56:32.160 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:56:32.160 INFO:teuthology.orchestra.run.smithi102.stdout: sudo /sbin/cephadm shell 2024-09-11T07:56:32.161 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:56:32.161 INFO:teuthology.orchestra.run.smithi102.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-11T07:56:32.161 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:56:32.161 INFO:teuthology.orchestra.run.smithi102.stdout: ceph telemetry on 2024-09-11T07:56:32.161 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:56:32.161 INFO:teuthology.orchestra.run.smithi102.stdout:For more information see: 2024-09-11T07:56:32.161 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:56:32.161 INFO:teuthology.orchestra.run.smithi102.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-09-11T07:56:32.161 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:56:32.161 INFO:teuthology.orchestra.run.smithi102.stdout:Bootstrap complete. 2024-09-11T07:56:32.189 INFO:tasks.cephadm:Fetching config... 2024-09-11T07:56:32.190 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-11T07:56:32.190 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-11T07:56:32.205 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-11T07:56:32.206 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-11T07:56:32.206 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-11T07:56:32.262 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-11T07:56:32.262 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-11T07:56:32.262 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-11T07:56:32.327 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-11T07:56:32.327 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-11T07:56:32.327 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-11T07:56:32.382 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-11T07:56:32.383 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCJazO2dLR6bekFDB1HF6xtX0o8yKt/esGCHLtfZ2JxW0lik2kFcVuCzE+dyPkvmeEW1jzh8GPCYe9yVHaF3kFic48Y2xTcvapMaxulEpfrCXRUUWMP3kdFe6FUq2p5aOMga9CaCikRKyQlYJ5cc/yoXJBcWTfk0cB1JnL12KUJKaD9zpPZt89PMihSvA9lMIEAuafgnid8nvl5tQpLEMk8HyPW7xGsm9pyO7AMfBPFngaoGlmAaEa1o/iOYMOgcIs8RX2vjgmOuMFu/P1a7B/U9ZU1ZMV7NPGwtjmNuR85Q2C5kvXXQrGPFRqSUfi/dSTJ21kkvGiuRYMVlJ004dt0BamTxy9OTlw3pBt4CiDJNyqjR6wssONrXb43jyiuUPKKDFCAfS8MEbAFPLSHIUx0irgXTRd89J8ThFhl2URuEj4DoYxIE4tnvpfxBQ77qIfgU3MuapC71Cqyk52XMaTby9IpgtWMLEbDCEhSuAY76x71a43MJQrraIbcuX2tHLU= ceph-2051892c-7013-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T07:56:32.467 INFO:teuthology.orchestra.run.smithi102.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCJazO2dLR6bekFDB1HF6xtX0o8yKt/esGCHLtfZ2JxW0lik2kFcVuCzE+dyPkvmeEW1jzh8GPCYe9yVHaF3kFic48Y2xTcvapMaxulEpfrCXRUUWMP3kdFe6FUq2p5aOMga9CaCikRKyQlYJ5cc/yoXJBcWTfk0cB1JnL12KUJKaD9zpPZt89PMihSvA9lMIEAuafgnid8nvl5tQpLEMk8HyPW7xGsm9pyO7AMfBPFngaoGlmAaEa1o/iOYMOgcIs8RX2vjgmOuMFu/P1a7B/U9ZU1ZMV7NPGwtjmNuR85Q2C5kvXXQrGPFRqSUfi/dSTJ21kkvGiuRYMVlJ004dt0BamTxy9OTlw3pBt4CiDJNyqjR6wssONrXb43jyiuUPKKDFCAfS8MEbAFPLSHIUx0irgXTRd89J8ThFhl2URuEj4DoYxIE4tnvpfxBQ77qIfgU3MuapC71Cqyk52XMaTby9IpgtWMLEbDCEhSuAY76x71a43MJQrraIbcuX2tHLU= ceph-2051892c-7013-11ef-bcea-c7b262605968 2024-09-11T07:56:32.479 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCJazO2dLR6bekFDB1HF6xtX0o8yKt/esGCHLtfZ2JxW0lik2kFcVuCzE+dyPkvmeEW1jzh8GPCYe9yVHaF3kFic48Y2xTcvapMaxulEpfrCXRUUWMP3kdFe6FUq2p5aOMga9CaCikRKyQlYJ5cc/yoXJBcWTfk0cB1JnL12KUJKaD9zpPZt89PMihSvA9lMIEAuafgnid8nvl5tQpLEMk8HyPW7xGsm9pyO7AMfBPFngaoGlmAaEa1o/iOYMOgcIs8RX2vjgmOuMFu/P1a7B/U9ZU1ZMV7NPGwtjmNuR85Q2C5kvXXQrGPFRqSUfi/dSTJ21kkvGiuRYMVlJ004dt0BamTxy9OTlw3pBt4CiDJNyqjR6wssONrXb43jyiuUPKKDFCAfS8MEbAFPLSHIUx0irgXTRd89J8ThFhl2URuEj4DoYxIE4tnvpfxBQ77qIfgU3MuapC71Cqyk52XMaTby9IpgtWMLEbDCEhSuAY76x71a43MJQrraIbcuX2tHLU= ceph-2051892c-7013-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T07:56:32.531 INFO:teuthology.orchestra.run.smithi146.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCJazO2dLR6bekFDB1HF6xtX0o8yKt/esGCHLtfZ2JxW0lik2kFcVuCzE+dyPkvmeEW1jzh8GPCYe9yVHaF3kFic48Y2xTcvapMaxulEpfrCXRUUWMP3kdFe6FUq2p5aOMga9CaCikRKyQlYJ5cc/yoXJBcWTfk0cB1JnL12KUJKaD9zpPZt89PMihSvA9lMIEAuafgnid8nvl5tQpLEMk8HyPW7xGsm9pyO7AMfBPFngaoGlmAaEa1o/iOYMOgcIs8RX2vjgmOuMFu/P1a7B/U9ZU1ZMV7NPGwtjmNuR85Q2C5kvXXQrGPFRqSUfi/dSTJ21kkvGiuRYMVlJ004dt0BamTxy9OTlw3pBt4CiDJNyqjR6wssONrXb43jyiuUPKKDFCAfS8MEbAFPLSHIUx0irgXTRd89J8ThFhl2URuEj4DoYxIE4tnvpfxBQ77qIfgU3MuapC71Cqyk52XMaTby9IpgtWMLEbDCEhSuAY76x71a43MJQrraIbcuX2tHLU= ceph-2051892c-7013-11ef-bcea-c7b262605968 2024-09-11T07:56:32.544 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-11T07:56:32.790 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:56:32.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4208325541' entity='client.admin' 2024-09-11T07:56:35.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.079+0000 7fabcc988640 1 -- 172.21.15.102:0/425557527 >> v1:172.21.15.102:6789/0 conn(0x7fabc80fc0c0 legacy=0x7fabc80fc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:35.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.079+0000 7fabcc988640 1 -- 172.21.15.102:0/425557527 shutdown_connections 2024-09-11T07:56:35.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.079+0000 7fabcc988640 1 -- 172.21.15.102:0/425557527 >> 172.21.15.102:0/425557527 conn(0x7fabc80f7880 msgr2=0x7fabc80f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:35.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.079+0000 7fabcc988640 1 -- 172.21.15.102:0/425557527 shutdown_connections 2024-09-11T07:56:35.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.079+0000 7fabcc988640 1 -- 172.21.15.102:0/425557527 wait complete. 2024-09-11T07:56:35.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.079+0000 7fabcc988640 1 Processor -- start 2024-09-11T07:56:35.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.079+0000 7fabcc988640 1 -- start start 2024-09-11T07:56:35.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.079+0000 7fabcc988640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fabc8071490 con 0x7fabc80fc0c0 2024-09-11T07:56:35.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.079+0000 7fabc659c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fabc80fc0c0 0x7fabc806fa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:42876/0 (socket says 172.21.15.102:42876) 2024-09-11T07:56:35.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.079+0000 7fabc659c640 1 -- 172.21.15.102:0/2285898971 learned_addr learned my addr 172.21.15.102:0/2285898971 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:35.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.080+0000 7fabbeffd640 1 -- 172.21.15.102:0/2285898971 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3013432612 0 0) 0x7fabc8071490 con 0x7fabc80fc0c0 2024-09-11T07:56:35.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.080+0000 7fabbeffd640 1 -- 172.21.15.102:0/2285898971 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabb40053f0 con 0x7fabc80fc0c0 2024-09-11T07:56:35.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.080+0000 7fabbeffd640 1 -- 172.21.15.102:0/2285898971 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 451681566 0 0) 0x7fabb40053f0 con 0x7fabc80fc0c0 2024-09-11T07:56:35.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.080+0000 7fabbeffd640 1 -- 172.21.15.102:0/2285898971 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fabc8071490 con 0x7fabc80fc0c0 2024-09-11T07:56:35.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.080+0000 7fabbeffd640 1 -- 172.21.15.102:0/2285898971 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fabb8001430 con 0x7fabc80fc0c0 2024-09-11T07:56:35.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.080+0000 7fabbeffd640 1 -- 172.21.15.102:0/2285898971 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 703865278 0 0) 0x7fabc8071490 con 0x7fabc80fc0c0 2024-09-11T07:56:35.119 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.080+0000 7fabbeffd640 1 -- 172.21.15.102:0/2285898971 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc8071620 con 0x7fabc80fc0c0 2024-09-11T07:56:35.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.080+0000 7fabcc988640 1 -- 172.21.15.102:0/2285898971 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc80718d0 con 0x7fabc80fc0c0 2024-09-11T07:56:35.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.080+0000 7fabcc988640 1 -- 172.21.15.102:0/2285898971 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc8070390 con 0x7fabc80fc0c0 2024-09-11T07:56:35.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.080+0000 7fabbeffd640 1 -- 172.21.15.102:0/2285898971 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7fabb80024a0 con 0x7fabc80fc0c0 2024-09-11T07:56:35.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.080+0000 7fabbeffd640 1 -- 172.21.15.102:0/2285898971 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fabb8003750 con 0x7fabc80fc0c0 2024-09-11T07:56:35.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.081+0000 7fabbeffd640 1 -- 172.21.15.102:0/2285898971 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 1600948874 0 0) 0x7fabb8003930 con 0x7fabc80fc0c0 2024-09-11T07:56:35.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.081+0000 7fabcc988640 1 -- 172.21.15.102:0/2285898971 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba8005040 con 0x7fabc80fc0c0 2024-09-11T07:56:35.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.081+0000 7fabbeffd640 1 -- 172.21.15.102:0/2285898971 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (unknown 4267545286 0 0) 0x7fabb8047c00 con 0x7fabc80fc0c0 2024-09-11T07:56:35.120 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.085+0000 7fabbeffd640 1 -- 172.21.15.102:0/2285898971 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fabb8016c60 con 0x7fabc80fc0c0 2024-09-11T07:56:35.248 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.247+0000 7fabcc988640 1 -- 172.21.15.102:0/2285898971 --> v1:172.21.15.102:6789/0 -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7faba8005870 con 0x7fabc80fc0c0 2024-09-11T07:56:35.252 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.251+0000 7fabbeffd640 1 -- 172.21.15.102:0/2285898971 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (unknown 3028693289 0 0) 0x7fabb8016660 con 0x7fabc80fc0c0 2024-09-11T07:56:35.254 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.253+0000 7fabcc988640 1 -- 172.21.15.102:0/2285898971 >> v1:172.21.15.102:6800/2950771867 conn(0x7fabb403a7d0 legacy=0x7fabb403cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:35.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.253+0000 7fabcc988640 1 -- 172.21.15.102:0/2285898971 >> v1:172.21.15.102:6789/0 conn(0x7fabc80fc0c0 legacy=0x7fabc806fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:35.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.253+0000 7fabcc988640 1 -- 172.21.15.102:0/2285898971 shutdown_connections 2024-09-11T07:56:35.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.253+0000 7fabcc988640 1 -- 172.21.15.102:0/2285898971 >> 172.21.15.102:0/2285898971 conn(0x7fabc80f7880 msgr2=0x7fabc80f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:35.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.253+0000 7fabcc988640 1 -- 172.21.15.102:0/2285898971 shutdown_connections 2024-09-11T07:56:35.255 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:35.253+0000 7fabcc988640 1 -- 172.21.15.102:0/2285898971 wait complete. 2024-09-11T07:56:35.896 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-11T07:56:35.896 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-11T07:56:36.142 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:56:36.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:36 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2285898971' entity='client.admin' 2024-09-11T07:56:37.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.883+0000 7f62b9c38640 1 -- 172.21.15.102:0/2692204311 >> v1:172.21.15.102:6789/0 conn(0x7f62b40fd720 legacy=0x7f62b40fdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:37.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.883+0000 7f62b9c38640 1 -- 172.21.15.102:0/2692204311 shutdown_connections 2024-09-11T07:56:37.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.883+0000 7f62b9c38640 1 -- 172.21.15.102:0/2692204311 >> 172.21.15.102:0/2692204311 conn(0x7f62b40f5f40 msgr2=0x7f62b40f8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:37.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.883+0000 7f62b9c38640 1 -- 172.21.15.102:0/2692204311 shutdown_connections 2024-09-11T07:56:37.887 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.883+0000 7f62b9c38640 1 -- 172.21.15.102:0/2692204311 wait complete. 2024-09-11T07:56:37.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.884+0000 7f62b9c38640 1 Processor -- start 2024-09-11T07:56:37.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.884+0000 7f62b9c38640 1 -- start start 2024-09-11T07:56:37.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.885+0000 7f62b9c38640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f62b410bee0 con 0x7f62b40fd720 2024-09-11T07:56:37.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.885+0000 7f62b37fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f62b40fd720 0x7f62b410b7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60060/0 (socket says 172.21.15.102:60060) 2024-09-11T07:56:37.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.885+0000 7f62b37fe640 1 -- 172.21.15.102:0/4130374324 learned_addr learned my addr 172.21.15.102:0/4130374324 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:37.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.885+0000 7f629bfff640 1 -- 172.21.15.102:0/4130374324 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3483895597 0 0) 0x7f62b410bee0 con 0x7f62b40fd720 2024-09-11T07:56:37.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.885+0000 7f629bfff640 1 -- 172.21.15.102:0/4130374324 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f629c0053f0 con 0x7f62b40fd720 2024-09-11T07:56:37.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.885+0000 7f629bfff640 1 -- 172.21.15.102:0/4130374324 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2583545020 0 0) 0x7f629c0053f0 con 0x7f62b40fd720 2024-09-11T07:56:37.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.885+0000 7f629bfff640 1 -- 172.21.15.102:0/4130374324 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f62b410bee0 con 0x7f62b40fd720 2024-09-11T07:56:37.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.885+0000 7f629bfff640 1 -- 172.21.15.102:0/4130374324 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f62a4001430 con 0x7f62b40fd720 2024-09-11T07:56:37.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.885+0000 7f629bfff640 1 -- 172.21.15.102:0/4130374324 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3275648109 0 0) 0x7f62b410bee0 con 0x7f62b40fd720 2024-09-11T07:56:37.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.885+0000 7f629bfff640 1 -- 172.21.15.102:0/4130374324 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b410c070 con 0x7f62b40fd720 2024-09-11T07:56:37.888 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.885+0000 7f62b9c38640 1 -- 172.21.15.102:0/4130374324 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b410c340 con 0x7f62b40fd720 2024-09-11T07:56:37.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.886+0000 7f62b9c38640 1 -- 172.21.15.102:0/4130374324 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b410ff50 con 0x7f62b40fd720 2024-09-11T07:56:37.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.886+0000 7f629bfff640 1 -- 172.21.15.102:0/4130374324 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f62a40024a0 con 0x7f62b40fd720 2024-09-11T07:56:37.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.886+0000 7f629bfff640 1 -- 172.21.15.102:0/4130374324 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f62a40036b0 con 0x7f62b40fd720 2024-09-11T07:56:37.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.886+0000 7f629bfff640 1 -- 172.21.15.102:0/4130374324 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 1600948874 0 0) 0x7f62a4003930 con 0x7f62b40fd720 2024-09-11T07:56:37.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.886+0000 7f62b9c38640 1 -- 172.21.15.102:0/4130374324 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a0005040 con 0x7f62b40fd720 2024-09-11T07:56:37.889 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.887+0000 7f629bfff640 1 -- 172.21.15.102:0/4130374324 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (unknown 4267545286 0 0) 0x7f62a4047ac0 con 0x7f62b40fd720 2024-09-11T07:56:37.892 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:37.891+0000 7f629bfff640 1 -- 172.21.15.102:0/4130374324 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f62a4049e60 con 0x7f62b40fd720 2024-09-11T07:56:38.049 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:38.048+0000 7f62b9c38640 1 -- 172.21.15.102:0/4130374324 --> v1:172.21.15.102:6800/2950771867 -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f62a0000bc0 con 0x7f629c03a7d0 2024-09-11T07:56:38.055 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:38.054+0000 7f629bfff640 1 -- 172.21.15.102:0/4130374324 <== mgr.14150 v1:172.21.15.102:6800/2950771867 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (unknown 0 0 0) 0x7f62a0000bc0 con 0x7f629c03a7d0 2024-09-11T07:56:38.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:38.056+0000 7f62b9c38640 1 -- 172.21.15.102:0/4130374324 >> v1:172.21.15.102:6800/2950771867 conn(0x7f629c03a7d0 legacy=0x7f629c03cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:38.057 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:38.056+0000 7f62b9c38640 1 -- 172.21.15.102:0/4130374324 >> v1:172.21.15.102:6789/0 conn(0x7f62b40fd720 legacy=0x7f62b410b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:38.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:38.056+0000 7f62b9c38640 1 -- 172.21.15.102:0/4130374324 shutdown_connections 2024-09-11T07:56:38.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:38.056+0000 7f62b9c38640 1 -- 172.21.15.102:0/4130374324 >> 172.21.15.102:0/4130374324 conn(0x7f62b40f5f40 msgr2=0x7f62b40f8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:38.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:38.056+0000 7f62b9c38640 1 -- 172.21.15.102:0/4130374324 shutdown_connections 2024-09-11T07:56:38.058 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:38.056+0000 7f62b9c38640 1 -- 172.21.15.102:0/4130374324 wait complete. 2024-09-11T07:56:38.893 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:38 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:38.894 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:38 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:38.894 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:38 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi102", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:56:38.894 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:38 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:38.894 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:38 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:38.894 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:38 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:38.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:38 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:38.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:38 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:38.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:38 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:38.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:38 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:38.919 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi146 2024-09-11T07:56:38.919 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-11T07:56:38.919 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/etc/ceph/ceph.conf 2024-09-11T07:56:38.940 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-11T07:56:38.940 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:56:38.999 INFO:tasks.cephadm:Adding host smithi146 to orchestrator... 2024-09-11T07:56:38.999 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph orch host add smithi146 2024-09-11T07:56:39.234 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:56:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:39 smithi102 ceph-mon[29849]: from='client.14174 v1:172.21.15.102:0/4130374324' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:56:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:39 smithi102 ceph-mon[29849]: Updating smithi102:/etc/ceph/ceph.conf 2024-09-11T07:56:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:39 smithi102 ceph-mon[29849]: Updating smithi102:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:56:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:39 smithi102 ceph-mon[29849]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:56:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:39 smithi102 ceph-mon[29849]: Updating smithi102:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:56:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:39 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:39 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:39 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:40.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.696+0000 7f7e6a3fd640 1 -- 172.21.15.102:0/4102038888 >> v1:172.21.15.102:6789/0 conn(0x7f7e64102180 legacy=0x7f7e64102550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:40.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.696+0000 7f7e6a3fd640 1 -- 172.21.15.102:0/4102038888 shutdown_connections 2024-09-11T07:56:40.728 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.696+0000 7f7e6a3fd640 1 -- 172.21.15.102:0/4102038888 >> 172.21.15.102:0/4102038888 conn(0x7f7e640f6300 msgr2=0x7f7e640f8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:40.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.696+0000 7f7e6a3fd640 1 -- 172.21.15.102:0/4102038888 shutdown_connections 2024-09-11T07:56:40.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.696+0000 7f7e6a3fd640 1 -- 172.21.15.102:0/4102038888 wait complete. 2024-09-11T07:56:40.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.696+0000 7f7e6a3fd640 1 Processor -- start 2024-09-11T07:56:40.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.697+0000 7f7e6a3fd640 1 -- start start 2024-09-11T07:56:40.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.697+0000 7f7e6a3fd640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7e64112360 con 0x7f7e64111880 2024-09-11T07:56:40.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.697+0000 7f7e63fff640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f7e64111880 0x7f7e64111c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60074/0 (socket says 172.21.15.102:60074) 2024-09-11T07:56:40.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.697+0000 7f7e63fff640 1 -- 172.21.15.102:0/2244810416 learned_addr learned my addr 172.21.15.102:0/2244810416 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:40.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.697+0000 7f7e60ff9640 1 -- 172.21.15.102:0/2244810416 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961078390 0 0) 0x7f7e64112360 con 0x7f7e64111880 2024-09-11T07:56:40.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.697+0000 7f7e60ff9640 1 -- 172.21.15.102:0/2244810416 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e4c0053f0 con 0x7f7e64111880 2024-09-11T07:56:40.729 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.697+0000 7f7e60ff9640 1 -- 172.21.15.102:0/2244810416 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3684176811 0 0) 0x7f7e4c0053f0 con 0x7f7e64111880 2024-09-11T07:56:40.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.697+0000 7f7e60ff9640 1 -- 172.21.15.102:0/2244810416 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7e64112360 con 0x7f7e64111880 2024-09-11T07:56:40.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.697+0000 7f7e60ff9640 1 -- 172.21.15.102:0/2244810416 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f7e54001060 con 0x7f7e64111880 2024-09-11T07:56:40.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.697+0000 7f7e60ff9640 1 -- 172.21.15.102:0/2244810416 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3363197464 0 0) 0x7f7e64112360 con 0x7f7e64111880 2024-09-11T07:56:40.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.697+0000 7f7e60ff9640 1 -- 172.21.15.102:0/2244810416 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e64115690 con 0x7f7e64111880 2024-09-11T07:56:40.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.697+0000 7f7e6a3fd640 1 -- 172.21.15.102:0/2244810416 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e64114680 con 0x7f7e64111880 2024-09-11T07:56:40.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.697+0000 7f7e6a3fd640 1 -- 172.21.15.102:0/2244810416 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e64114ad0 con 0x7f7e64111880 2024-09-11T07:56:40.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.698+0000 7f7e60ff9640 1 -- 172.21.15.102:0/2244810416 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f7e540022c0 con 0x7f7e64111880 2024-09-11T07:56:40.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.698+0000 7f7e60ff9640 1 -- 172.21.15.102:0/2244810416 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f7e54003540 con 0x7f7e64111880 2024-09-11T07:56:40.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.698+0000 7f7e60ff9640 1 -- 172.21.15.102:0/2244810416 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 1600948874 0 0) 0x7f7e540036a0 con 0x7f7e64111880 2024-09-11T07:56:40.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.698+0000 7f7e6a3fd640 1 -- 172.21.15.102:0/2244810416 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e50005010 con 0x7f7e64111880 2024-09-11T07:56:40.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.698+0000 7f7e60ff9640 1 -- 172.21.15.102:0/2244810416 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (unknown 4267545286 0 0) 0x7f7e540479c0 con 0x7f7e64111880 2024-09-11T07:56:40.730 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.703+0000 7f7e60ff9640 1 -- 172.21.15.102:0/2244810416 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f7e54016980 con 0x7f7e64111880 2024-09-11T07:56:40.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:40.862+0000 7f7e6a3fd640 1 -- 172.21.15.102:0/2244810416 --> v1:172.21.15.102:6800/2950771867 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi146", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e50000b90 con 0x7f7e4c03a800 2024-09-11T07:56:41.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:41 smithi102 ceph-mon[29849]: from='client.14176 v1:172.21.15.102:0/2244810416' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi146", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:56:42.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:42 smithi102 ceph-mon[29849]: Deploying cephadm binary to smithi146 2024-09-11T07:56:43.479 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:43.478+0000 7f7e60ff9640 1 -- 172.21.15.102:0/2244810416 <== mgr.14150 v1:172.21.15.102:6800/2950771867 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (unknown 0 0 1110543527) 0x7f7e50000b90 con 0x7f7e4c03a800 2024-09-11T07:56:43.480 INFO:teuthology.orchestra.run.smithi102.stdout:Added host 'smithi146' with addr '172.21.15.146' 2024-09-11T07:56:43.481 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:43.480+0000 7f7e6a3fd640 1 -- 172.21.15.102:0/2244810416 >> v1:172.21.15.102:6800/2950771867 conn(0x7f7e4c03a800 legacy=0x7f7e4c03ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:43.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:43.480+0000 7f7e6a3fd640 1 -- 172.21.15.102:0/2244810416 >> v1:172.21.15.102:6789/0 conn(0x7f7e64111880 legacy=0x7f7e64111c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:43.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:43.480+0000 7f7e6a3fd640 1 -- 172.21.15.102:0/2244810416 shutdown_connections 2024-09-11T07:56:43.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:43.480+0000 7f7e6a3fd640 1 -- 172.21.15.102:0/2244810416 >> 172.21.15.102:0/2244810416 conn(0x7f7e640f6300 msgr2=0x7f7e64103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:43.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:43.481+0000 7f7e6a3fd640 1 -- 172.21.15.102:0/2244810416 shutdown_connections 2024-09-11T07:56:43.482 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:43.481+0000 7f7e6a3fd640 1 -- 172.21.15.102:0/2244810416 wait complete. 2024-09-11T07:56:44.047 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T07:56:44.280 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:56:44.667 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:44 smithi102 ceph-mon[29849]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:56:44.667 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:44 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:44.667 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:44 smithi102 ceph-mon[29849]: Added host smithi146 2024-09-11T07:56:44.667 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:44 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:44.667 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:44 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:44.667 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:44 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:45.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.816+0000 7fd0c8a26640 1 -- 172.21.15.102:0/307429077 >> v1:172.21.15.102:6789/0 conn(0x7fd0c40fd100 legacy=0x7fd0c40fd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:45.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.817+0000 7fd0c8a26640 1 -- 172.21.15.102:0/307429077 shutdown_connections 2024-09-11T07:56:45.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.817+0000 7fd0c8a26640 1 -- 172.21.15.102:0/307429077 >> 172.21.15.102:0/307429077 conn(0x7fd0c40f6be0 msgr2=0x7fd0c40f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:45.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.817+0000 7fd0c8a26640 1 -- 172.21.15.102:0/307429077 shutdown_connections 2024-09-11T07:56:45.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.817+0000 7fd0c8a26640 1 -- 172.21.15.102:0/307429077 wait complete. 2024-09-11T07:56:45.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.817+0000 7fd0c8a26640 1 Processor -- start 2024-09-11T07:56:45.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.817+0000 7fd0c8a26640 1 -- start start 2024-09-11T07:56:45.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.817+0000 7fd0c8a26640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd0c4112010 con 0x7fd0c40fd100 2024-09-11T07:56:45.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.817+0000 7fd0c259c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd0c40fd100 0x7fd0c4111900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60098/0 (socket says 172.21.15.102:60098) 2024-09-11T07:56:45.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.817+0000 7fd0c259c640 1 -- 172.21.15.102:0/2052645394 learned_addr learned my addr 172.21.15.102:0/2052645394 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:45.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.819+0000 7fd0baffd640 1 -- 172.21.15.102:0/2052645394 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 574805797 0 0) 0x7fd0c4112010 con 0x7fd0c40fd100 2024-09-11T07:56:45.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.819+0000 7fd0baffd640 1 -- 172.21.15.102:0/2052645394 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0b00053f0 con 0x7fd0c40fd100 2024-09-11T07:56:45.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.819+0000 7fd0baffd640 1 -- 172.21.15.102:0/2052645394 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1133784624 0 0) 0x7fd0b00053f0 con 0x7fd0c40fd100 2024-09-11T07:56:45.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.819+0000 7fd0baffd640 1 -- 172.21.15.102:0/2052645394 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd0c4112010 con 0x7fd0c40fd100 2024-09-11T07:56:45.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.819+0000 7fd0baffd640 1 -- 172.21.15.102:0/2052645394 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fd0b4001550 con 0x7fd0c40fd100 2024-09-11T07:56:45.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.821+0000 7fd0baffd640 1 -- 172.21.15.102:0/2052645394 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4253565460 0 0) 0x7fd0c4112010 con 0x7fd0c40fd100 2024-09-11T07:56:45.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.821+0000 7fd0baffd640 1 -- 172.21.15.102:0/2052645394 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c41121a0 con 0x7fd0c40fd100 2024-09-11T07:56:45.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.821+0000 7fd0c8a26640 1 -- 172.21.15.102:0/2052645394 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c4112470 con 0x7fd0c40fd100 2024-09-11T07:56:45.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.821+0000 7fd0c8a26640 1 -- 172.21.15.102:0/2052645394 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c41148a0 con 0x7fd0c40fd100 2024-09-11T07:56:45.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.821+0000 7fd0baffd640 1 -- 172.21.15.102:0/2052645394 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7fd0b40011e0 con 0x7fd0c40fd100 2024-09-11T07:56:45.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.821+0000 7fd0baffd640 1 -- 172.21.15.102:0/2052645394 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7fd0b4003540 con 0x7fd0c40fd100 2024-09-11T07:56:45.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.822+0000 7fd0baffd640 1 -- 172.21.15.102:0/2052645394 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 1600948874 0 0) 0x7fd0b40036c0 con 0x7fd0c40fd100 2024-09-11T07:56:45.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.822+0000 7fd0c8a26640 1 -- 172.21.15.102:0/2052645394 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a4005010 con 0x7fd0c40fd100 2024-09-11T07:56:45.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.822+0000 7fd0baffd640 1 -- 172.21.15.102:0/2052645394 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (unknown 4267545286 0 0) 0x7fd0b4047760 con 0x7fd0c40fd100 2024-09-11T07:56:45.867 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.827+0000 7fd0baffd640 1 -- 172.21.15.102:0/2052645394 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fd0b4003a60 con 0x7fd0c40fd100 2024-09-11T07:56:45.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:45 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:45.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.987+0000 7fd0c8a26640 1 -- 172.21.15.102:0/2052645394 --> v1:172.21.15.102:6800/2950771867 -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd0a4000b90 con 0x7fd0b003a8d0 2024-09-11T07:56:45.989 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.989+0000 7fd0baffd640 1 -- 172.21.15.102:0/2052645394 <== mgr.14150 v1:172.21.15.102:6800/2950771867 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (unknown 0 0 3643627695) 0x7fd0a4000b90 con 0x7fd0b003a8d0 2024-09-11T07:56:45.990 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T07:56:45.990 INFO:teuthology.orchestra.run.smithi102.stdout:[{"addr": "172.21.15.102", "hostname": "smithi102", "labels": [], "status": ""}, {"addr": "172.21.15.146", "hostname": "smithi146", "labels": [], "status": ""}] 2024-09-11T07:56:45.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.990+0000 7fd0c8a26640 1 -- 172.21.15.102:0/2052645394 >> v1:172.21.15.102:6800/2950771867 conn(0x7fd0b003a8d0 legacy=0x7fd0b003cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:45.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.990+0000 7fd0c8a26640 1 -- 172.21.15.102:0/2052645394 >> v1:172.21.15.102:6789/0 conn(0x7fd0c40fd100 legacy=0x7fd0c4111900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:45.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.991+0000 7fd0c8a26640 1 -- 172.21.15.102:0/2052645394 shutdown_connections 2024-09-11T07:56:45.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.991+0000 7fd0c8a26640 1 -- 172.21.15.102:0/2052645394 >> 172.21.15.102:0/2052645394 conn(0x7fd0c40f6be0 msgr2=0x7fd0c40f8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:45.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.991+0000 7fd0c8a26640 1 -- 172.21.15.102:0/2052645394 shutdown_connections 2024-09-11T07:56:45.992 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:45.991+0000 7fd0c8a26640 1 -- 172.21.15.102:0/2052645394 wait complete. 2024-09-11T07:56:46.576 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-11T07:56:46.577 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-11T07:56:46.809 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:56:47.072 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:46 smithi102 ceph-mon[29849]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:56:47.900 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:47 smithi102 ceph-mon[29849]: from='client.14178 v1:172.21.15.102:0/2052645394' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:56:47.900 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:47 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:47.900 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:47 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:47.900 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:47 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:47.900 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:47 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:47.900 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:47 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi146", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:56:47.900 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:47 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:47.900 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:47 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:48.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.407+0000 7f021e37c640 1 -- 172.21.15.102:0/1428650722 >> v1:172.21.15.102:6789/0 conn(0x7f0218100ff0 legacy=0x7f02181013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:48.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.407+0000 7f021e37c640 1 -- 172.21.15.102:0/1428650722 shutdown_connections 2024-09-11T07:56:48.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.407+0000 7f021e37c640 1 -- 172.21.15.102:0/1428650722 >> 172.21.15.102:0/1428650722 conn(0x7f0218070fb0 msgr2=0x7f02180713b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:48.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.407+0000 7f021e37c640 1 -- 172.21.15.102:0/1428650722 shutdown_connections 2024-09-11T07:56:48.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.407+0000 7f021e37c640 1 -- 172.21.15.102:0/1428650722 wait complete. 2024-09-11T07:56:48.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.407+0000 7f021e37c640 1 Processor -- start 2024-09-11T07:56:48.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.407+0000 7f021e37c640 1 -- start start 2024-09-11T07:56:48.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.408+0000 7f021e37c640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f02180fd3a0 con 0x7f0218100ff0 2024-09-11T07:56:48.412 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.408+0000 7f0217fff640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f0218100ff0 0x7f02180fcc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:53996/0 (socket says 172.21.15.102:53996) 2024-09-11T07:56:48.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.408+0000 7f0217fff640 1 -- 172.21.15.102:0/3700124607 learned_addr learned my addr 172.21.15.102:0/3700124607 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:56:48.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.408+0000 7f02157fa640 1 -- 172.21.15.102:0/3700124607 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 723823693 0 0) 0x7f02180fd3a0 con 0x7f0218100ff0 2024-09-11T07:56:48.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.408+0000 7f02157fa640 1 -- 172.21.15.102:0/3700124607 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02040053f0 con 0x7f0218100ff0 2024-09-11T07:56:48.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.408+0000 7f02157fa640 1 -- 172.21.15.102:0/3700124607 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3485712652 0 0) 0x7f02040053f0 con 0x7f0218100ff0 2024-09-11T07:56:48.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.408+0000 7f02157fa640 1 -- 172.21.15.102:0/3700124607 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f02180fd3a0 con 0x7f0218100ff0 2024-09-11T07:56:48.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.408+0000 7f02157fa640 1 -- 172.21.15.102:0/3700124607 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f0208002020 con 0x7f0218100ff0 2024-09-11T07:56:48.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.408+0000 7f02157fa640 1 -- 172.21.15.102:0/3700124607 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1180505155 0 0) 0x7f02180fd3a0 con 0x7f0218100ff0 2024-09-11T07:56:48.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.408+0000 7f02157fa640 1 -- 172.21.15.102:0/3700124607 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02180fd530 con 0x7f0218100ff0 2024-09-11T07:56:48.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.408+0000 7f021e37c640 1 -- 172.21.15.102:0/3700124607 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02180fd7e0 con 0x7f0218100ff0 2024-09-11T07:56:48.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.408+0000 7f02157fa640 1 -- 172.21.15.102:0/3700124607 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f02080016a0 con 0x7f0218100ff0 2024-09-11T07:56:48.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.408+0000 7f021e37c640 1 -- 172.21.15.102:0/3700124607 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0218116a20 con 0x7f0218100ff0 2024-09-11T07:56:48.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.408+0000 7f02157fa640 1 -- 172.21.15.102:0/3700124607 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f02080037a0 con 0x7f0218100ff0 2024-09-11T07:56:48.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.409+0000 7f02157fa640 1 -- 172.21.15.102:0/3700124607 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 1600948874 0 0) 0x7f0208003a40 con 0x7f0218100ff0 2024-09-11T07:56:48.413 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.409+0000 7f021e37c640 1 -- 172.21.15.102:0/3700124607 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02180fc760 con 0x7f0218100ff0 2024-09-11T07:56:48.414 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.410+0000 7f02157fa640 1 -- 172.21.15.102:0/3700124607 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (unknown 4267545286 0 0) 0x7f0208047c10 con 0x7f0218100ff0 2024-09-11T07:56:48.415 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.414+0000 7f02157fa640 1 -- 172.21.15.102:0/3700124607 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f0208016b70 con 0x7f0218100ff0 2024-09-11T07:56:48.580 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.579+0000 7f021e37c640 1 -- 172.21.15.102:0/3700124607 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f021804b720 con 0x7f0218100ff0 2024-09-11T07:56:48.827 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.825+0000 7f02157fa640 1 -- 172.21.15.102:0/3700124607 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (unknown 3126668360 0 0) 0x7f0208016570 con 0x7f0218100ff0 2024-09-11T07:56:48.827 INFO:teuthology.orchestra.run.smithi102.stderr:adjusted tunables profile to default 2024-09-11T07:56:48.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.828+0000 7f021e37c640 1 -- 172.21.15.102:0/3700124607 >> v1:172.21.15.102:6800/2950771867 conn(0x7f020403a730 legacy=0x7f020403cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:48.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.828+0000 7f021e37c640 1 -- 172.21.15.102:0/3700124607 >> v1:172.21.15.102:6789/0 conn(0x7f0218100ff0 legacy=0x7f02180fcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:48.829 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.828+0000 7f021e37c640 1 -- 172.21.15.102:0/3700124607 shutdown_connections 2024-09-11T07:56:48.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.828+0000 7f021e37c640 1 -- 172.21.15.102:0/3700124607 >> 172.21.15.102:0/3700124607 conn(0x7f0218070fb0 msgr2=0x7f0218064930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:48.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.828+0000 7f021e37c640 1 -- 172.21.15.102:0/3700124607 shutdown_connections 2024-09-11T07:56:48.830 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:56:48.828+0000 7f021e37c640 1 -- 172.21.15.102:0/3700124607 wait complete. 2024-09-11T07:56:48.838 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:48 smithi102 ceph-mon[29849]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:56:48.839 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:48 smithi102 ceph-mon[29849]: Updating smithi146:/etc/ceph/ceph.conf 2024-09-11T07:56:48.839 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3700124607' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T07:56:48.839 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:48 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:48.839 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:48 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:48.839 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:48 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:49.433 INFO:tasks.cephadm:Adding mon.a on smithi102 2024-09-11T07:56:49.433 INFO:tasks.cephadm:Adding mon.c on smithi102 2024-09-11T07:56:49.433 INFO:tasks.cephadm:Adding mon.b on smithi146 2024-09-11T07:56:49.433 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi102:[v1:172.21.15.102:6789]=a;smithi102:[v1:172.21.15.102:6790]=c;smithi146:[v1:172.21.15.146:6789]=b' 2024-09-11T07:56:49.670 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:56:50.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:49 smithi102 ceph-mon[29849]: Updating smithi146:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:56:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:49 smithi102 ceph-mon[29849]: Updating smithi146:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:56:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:49 smithi102 ceph-mon[29849]: Updating smithi146:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:56:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:49 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3700124607' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T07:56:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:49 smithi102 ceph-mon[29849]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:56:51.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:50 smithi102 ceph-mon[29849]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:56:51.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.424+0000 7efd81c8c640 1 -- 172.21.15.146:0/1296968300 >> v1:172.21.15.102:6789/0 conn(0x7efd7c101000 legacy=0x7efd7c1013d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:51.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.424+0000 7efd81c8c640 1 -- 172.21.15.146:0/1296968300 shutdown_connections 2024-09-11T07:56:51.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.424+0000 7efd81c8c640 1 -- 172.21.15.146:0/1296968300 >> 172.21.15.146:0/1296968300 conn(0x7efd7c0f6bf0 msgr2=0x7efd7c0f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:51.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.425+0000 7efd81c8c640 1 -- 172.21.15.146:0/1296968300 shutdown_connections 2024-09-11T07:56:51.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.425+0000 7efd81c8c640 1 -- 172.21.15.146:0/1296968300 wait complete. 2024-09-11T07:56:51.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.425+0000 7efd81c8c640 1 Processor -- start 2024-09-11T07:56:51.427 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.425+0000 7efd81c8c640 1 -- start start 2024-09-11T07:56:51.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.426+0000 7efd81c8c640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7efd7c10cd20 con 0x7efd7c101000 2024-09-11T07:56:51.428 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.426+0000 7efd7b7fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7efd7c101000 0x7efd7c10c610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.146:34132/0 (socket says 172.21.15.146:34132) 2024-09-11T07:56:51.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.426+0000 7efd7b7fe640 1 -- 172.21.15.146:0/1903928884 learned_addr learned my addr 172.21.15.146:0/1903928884 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-11T07:56:51.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.426+0000 7efd73fff640 1 -- 172.21.15.146:0/1903928884 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2004658213 0 0) 0x7efd7c10cd20 con 0x7efd7c101000 2024-09-11T07:56:51.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.426+0000 7efd73fff640 1 -- 172.21.15.146:0/1903928884 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd680053f0 con 0x7efd7c101000 2024-09-11T07:56:51.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.426+0000 7efd73fff640 1 -- 172.21.15.146:0/1903928884 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 529784515 0 0) 0x7efd680053f0 con 0x7efd7c101000 2024-09-11T07:56:51.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.426+0000 7efd73fff640 1 -- 172.21.15.146:0/1903928884 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efd7c10cd20 con 0x7efd7c101000 2024-09-11T07:56:51.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.426+0000 7efd73fff640 1 -- 172.21.15.146:0/1903928884 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7efd6c002020 con 0x7efd7c101000 2024-09-11T07:56:51.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.426+0000 7efd73fff640 1 -- 172.21.15.146:0/1903928884 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2480540809 0 0) 0x7efd7c10cd20 con 0x7efd7c101000 2024-09-11T07:56:51.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.427+0000 7efd73fff640 1 -- 172.21.15.146:0/1903928884 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7c10ceb0 con 0x7efd7c101000 2024-09-11T07:56:51.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.427+0000 7efd81c8c640 1 -- 172.21.15.146:0/1903928884 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7c10d160 con 0x7efd7c101000 2024-09-11T07:56:51.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.427+0000 7efd81c8c640 1 -- 172.21.15.146:0/1903928884 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7efd7c114700 con 0x7efd7c101000 2024-09-11T07:56:51.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.427+0000 7efd73fff640 1 -- 172.21.15.146:0/1903928884 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7efd6c001590 con 0x7efd7c101000 2024-09-11T07:56:51.429 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.427+0000 7efd73fff640 1 -- 172.21.15.146:0/1903928884 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7efd6c003530 con 0x7efd7c101000 2024-09-11T07:56:51.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.427+0000 7efd73fff640 1 -- 172.21.15.146:0/1903928884 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 1600948874 0 0) 0x7efd6c003750 con 0x7efd7c101000 2024-09-11T07:56:51.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.427+0000 7efd81c8c640 1 -- 172.21.15.146:0/1903928884 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd5c005040 con 0x7efd7c101000 2024-09-11T07:56:51.430 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.428+0000 7efd73fff640 1 -- 172.21.15.146:0/1903928884 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (unknown 1709346042 0 0) 0x7efd6c047ab0 con 0x7efd7c101000 2024-09-11T07:56:51.433 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.432+0000 7efd73fff640 1 -- 172.21.15.146:0/1903928884 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7efd6c016a60 con 0x7efd7c101000 2024-09-11T07:56:51.599 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.597+0000 7efd81c8c640 1 -- 172.21.15.146:0/1903928884 --> v1:172.21.15.102:6800/2950771867 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi102:[v1:172.21.15.102:6789]=a;smithi102:[v1:172.21.15.102:6790]=c;smithi146:[v1:172.21.15.146:6789]=b", "target": ["mon-mgr", ""]}) v1 -- 0x7efd5c000ca0 con 0x7efd6803a7d0 2024-09-11T07:56:51.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.602+0000 7efd73fff640 1 -- 172.21.15.146:0/1903928884 <== mgr.14150 v1:172.21.15.102:6800/2950771867 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 3265049985) 0x7efd5c000ca0 con 0x7efd6803a7d0 2024-09-11T07:56:51.604 INFO:teuthology.orchestra.run.smithi146.stdout:Scheduled mon update... 2024-09-11T07:56:51.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.604+0000 7efd81c8c640 1 -- 172.21.15.146:0/1903928884 >> v1:172.21.15.102:6800/2950771867 conn(0x7efd6803a7d0 legacy=0x7efd6803cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:51.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.604+0000 7efd81c8c640 1 -- 172.21.15.146:0/1903928884 >> v1:172.21.15.102:6789/0 conn(0x7efd7c101000 legacy=0x7efd7c10c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:51.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.604+0000 7efd81c8c640 1 -- 172.21.15.146:0/1903928884 shutdown_connections 2024-09-11T07:56:51.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.604+0000 7efd81c8c640 1 -- 172.21.15.146:0/1903928884 >> 172.21.15.146:0/1903928884 conn(0x7efd7c0f6bf0 msgr2=0x7efd7c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:51.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.604+0000 7efd81c8c640 1 -- 172.21.15.146:0/1903928884 shutdown_connections 2024-09-11T07:56:51.606 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:51.604+0000 7efd81c8c640 1 -- 172.21.15.146:0/1903928884 wait complete. 2024-09-11T07:56:52.163 DEBUG:teuthology.orchestra.run.smithi102:mon.c> sudo journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@mon.c.service 2024-09-11T07:56:52.166 DEBUG:teuthology.orchestra.run.smithi146:mon.b> sudo journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@mon.b.service 2024-09-11T07:56:52.169 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T07:56:52.169 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T07:56:52.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:52 smithi102 ceph-mon[29849]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:56:52.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:52 smithi102 ceph-mon[29849]: from='client.14182 v1:172.21.15.146:0/1903928884' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi102:[v1:172.21.15.102:6789]=a;smithi102:[v1:172.21.15.102:6790]=c;smithi146:[v1:172.21.15.146:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:56:52.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:52 smithi102 ceph-mon[29849]: Saving service mon spec with placement smithi102:[v1:172.21.15.102:6789]=a;smithi102:[v1:172.21.15.102:6790]=c;smithi146:[v1:172.21.15.146:6789]=b;count:3 2024-09-11T07:56:52.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:52 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:52.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:52 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:56:52.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:52 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:52.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:52 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:56:52.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:52 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:56:52.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:52 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:56:52.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:52 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:56:52.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:52 smithi102 ceph-mon[29849]: Deploying daemon mon.b on smithi146 2024-09-11T07:56:53.276 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:56:54.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:56:54 smithi102 ceph-mon[29849]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:56:56.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.030+0000 7f2c19814640 1 -- 172.21.15.146:0/2965411077 >> v1:172.21.15.102:6789/0 conn(0x7f2c14101010 legacy=0x7f2c141013e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:56.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.030+0000 7f2c19814640 1 -- 172.21.15.146:0/2965411077 shutdown_connections 2024-09-11T07:56:56.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.030+0000 7f2c19814640 1 -- 172.21.15.146:0/2965411077 >> 172.21.15.146:0/2965411077 conn(0x7f2c140f6be0 msgr2=0x7f2c140f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:56.043 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.030+0000 7f2c19814640 1 -- 172.21.15.146:0/2965411077 shutdown_connections 2024-09-11T07:56:56.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.030+0000 7f2c19814640 1 -- 172.21.15.146:0/2965411077 wait complete. 2024-09-11T07:56:56.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.031+0000 7f2c19814640 1 Processor -- start 2024-09-11T07:56:56.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.031+0000 7f2c19814640 1 -- start start 2024-09-11T07:56:56.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.031+0000 7f2c19814640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2c14111d90 con 0x7f2c14101010 2024-09-11T07:56:56.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.031+0000 7f2c12ffd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2c14101010 0x7f2c14111680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.146:34530/0 (socket says 172.21.15.146:34530) 2024-09-11T07:56:56.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.031+0000 7f2c12ffd640 1 -- 172.21.15.146:0/708630367 learned_addr learned my addr 172.21.15.146:0/708630367 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-11T07:56:56.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.031+0000 7f2bfb7fe640 1 -- 172.21.15.146:0/708630367 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2293679023 0 0) 0x7f2c14111d90 con 0x7f2c14101010 2024-09-11T07:56:56.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.031+0000 7f2bfb7fe640 1 -- 172.21.15.146:0/708630367 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bfc0053f0 con 0x7f2c14101010 2024-09-11T07:56:56.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.032+0000 7f2bfb7fe640 1 -- 172.21.15.146:0/708630367 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3342174027 0 0) 0x7f2bfc0053f0 con 0x7f2c14101010 2024-09-11T07:56:56.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.032+0000 7f2bfb7fe640 1 -- 172.21.15.146:0/708630367 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2c14111d90 con 0x7f2c14101010 2024-09-11T07:56:56.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.032+0000 7f2bfb7fe640 1 -- 172.21.15.146:0/708630367 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f2c04002020 con 0x7f2c14101010 2024-09-11T07:56:56.044 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.032+0000 7f2bfb7fe640 1 -- 172.21.15.146:0/708630367 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1233700903 0 0) 0x7f2c14111d90 con 0x7f2c14101010 2024-09-11T07:56:56.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.032+0000 7f2bfb7fe640 1 -- 172.21.15.146:0/708630367 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c14111f20 con 0x7f2c14101010 2024-09-11T07:56:56.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.032+0000 7f2c19814640 1 -- 172.21.15.146:0/708630367 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c141121f0 con 0x7f2c14101010 2024-09-11T07:56:56.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.032+0000 7f2c19814640 1 -- 172.21.15.146:0/708630367 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c14114710 con 0x7f2c14101010 2024-09-11T07:56:56.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.032+0000 7f2bfb7fe640 1 -- 172.21.15.146:0/708630367 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f2c040015d0 con 0x7f2c14101010 2024-09-11T07:56:56.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.032+0000 7f2bfb7fe640 1 -- 172.21.15.146:0/708630367 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1571286715 0 0) 0x7f2c040035d0 con 0x7f2c14101010 2024-09-11T07:56:56.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.033+0000 7f2c19814640 1 -- 172.21.15.146:0/708630367 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c00005010 con 0x7f2c14101010 2024-09-11T07:56:56.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.033+0000 7f2bfb7fe640 1 -- 172.21.15.146:0/708630367 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 1600948874 0 0) 0x7f2c04002250 con 0x7f2c14101010 2024-09-11T07:56:56.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.033+0000 7f2bfb7fe640 1 -- 172.21.15.146:0/708630367 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (unknown 1709346042 0 0) 0x7f2c04047af0 con 0x7f2c14101010 2024-09-11T07:56:56.045 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.038+0000 7f2bfb7fe640 1 -- 172.21.15.146:0/708630367 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f2c04016070 con 0x7f2c14101010 2024-09-11T07:56:56.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.259+0000 7f2c19814640 1 -- 172.21.15.146:0/708630367 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2c000052c0 con 0x7f2c14101010 2024-09-11T07:56:56.261 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.259+0000 7f2bfb7fe640 1 -- 172.21.15.146:0/708630367 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+683 (unknown 2237029548 0 2904792209) 0x7f2c04015a70 con 0x7f2c14101010 2024-09-11T07:56:56.262 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:56:56.262 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":1,"fsid":"2051892c-7013-11ef-bcea-c7b262605968","modified":"2024-09-11T07:55:28.194357Z","created":"2024-09-11T07:55:28.194357Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T07:56:56.262 INFO:teuthology.orchestra.run.smithi146.stderr:dumped monmap epoch 1 2024-09-11T07:56:56.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.262+0000 7f2c19814640 1 -- 172.21.15.146:0/708630367 >> v1:172.21.15.102:6800/2950771867 conn(0x7f2bfc03a7a0 legacy=0x7f2bfc03cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:56.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.262+0000 7f2c19814640 1 -- 172.21.15.146:0/708630367 >> v1:172.21.15.102:6789/0 conn(0x7f2c14101010 legacy=0x7f2c14111680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:56:56.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.262+0000 7f2c19814640 1 -- 172.21.15.146:0/708630367 shutdown_connections 2024-09-11T07:56:56.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.262+0000 7f2c19814640 1 -- 172.21.15.146:0/708630367 >> 172.21.15.146:0/708630367 conn(0x7f2c140f6be0 msgr2=0x7f2c140654e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:56:56.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.262+0000 7f2c19814640 1 -- 172.21.15.146:0/708630367 shutdown_connections 2024-09-11T07:56:56.264 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:56:56.262+0000 7f2c19814640 1 -- 172.21.15.146:0/708630367 wait complete. 2024-09-11T07:56:58.038 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T07:56:58.039 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T07:56:58.292 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:57:01.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.598+0000 7f2e3215e640 1 -- 172.21.15.146:0/3845523039 >> v1:172.21.15.102:6789/0 conn(0x7f2e2c1011c0 legacy=0x7f2e2c101590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:01.601 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.598+0000 7f2e3215e640 1 -- 172.21.15.146:0/3845523039 shutdown_connections 2024-09-11T07:57:01.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.598+0000 7f2e3215e640 1 -- 172.21.15.146:0/3845523039 >> 172.21.15.146:0/3845523039 conn(0x7f2e2c071150 msgr2=0x7f2e2c071550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:57:01.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.598+0000 7f2e3215e640 1 -- 172.21.15.146:0/3845523039 shutdown_connections 2024-09-11T07:57:01.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.598+0000 7f2e3215e640 1 -- 172.21.15.146:0/3845523039 wait complete. 2024-09-11T07:57:01.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.599+0000 7f2e3215e640 1 Processor -- start 2024-09-11T07:57:01.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.599+0000 7f2e3215e640 1 -- start start 2024-09-11T07:57:01.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.600+0000 7f2e3215e640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2e2c117e90 con 0x7f2e2c1011c0 2024-09-11T07:57:01.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.600+0000 7f2e3215e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2e2c119030 con 0x7f2e2c112070 2024-09-11T07:57:01.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.600+0000 7f2e2b7fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2e2c1011c0 0x7f2e2c111960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.146:34570/0 (socket says 172.21.15.146:34570) 2024-09-11T07:57:01.602 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.600+0000 7f2e2b7fe640 1 -- 172.21.15.146:0/1246313149 learned_addr learned my addr 172.21.15.146:0/1246313149 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-11T07:57:01.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.600+0000 7f2e13fff640 1 -- 172.21.15.146:0/1246313149 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 833490095 0 0) 0x7f2e2c117e90 con 0x7f2e2c1011c0 2024-09-11T07:57:01.603 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.600+0000 7f2e13fff640 1 -- 172.21.15.146:0/1246313149 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e14005590 con 0x7f2e2c1011c0 2024-09-11T07:57:01.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.600+0000 7f2e13fff640 1 -- 172.21.15.146:0/1246313149 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 177384774 0 0) 0x7f2e14005590 con 0x7f2e2c1011c0 2024-09-11T07:57:01.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.600+0000 7f2e13fff640 1 -- 172.21.15.146:0/1246313149 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2e2c117e90 con 0x7f2e2c1011c0 2024-09-11T07:57:01.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.600+0000 7f2e13fff640 1 -- 172.21.15.146:0/1246313149 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 239+0+0 (unknown 2620338839 0 0) 0x7f2e1c002ff0 con 0x7f2e2c1011c0 2024-09-11T07:57:01.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.601+0000 7f2e13fff640 1 -- 172.21.15.146:0/1246313149 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2534361025 0 0) 0x7f2e2c117e90 con 0x7f2e2c1011c0 2024-09-11T07:57:01.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.601+0000 7f2e13fff640 1 -- 172.21.15.146:0/1246313149 >> v1:172.21.15.146:6789/0 conn(0x7f2e2c112070 legacy=0x7f2e2c116770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:01.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.601+0000 7f2e13fff640 1 -- 172.21.15.146:0/1246313149 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c11a1d0 con 0x7f2e2c1011c0 2024-09-11T07:57:01.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.601+0000 7f2e3215e640 1 -- 172.21.15.146:0/1246313149 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e2c1191c0 con 0x7f2e2c1011c0 2024-09-11T07:57:01.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.601+0000 7f2e3215e640 1 -- 172.21.15.146:0/1246313149 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2c1196d0 con 0x7f2e2c1011c0 2024-09-11T07:57:01.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.601+0000 7f2e13fff640 1 -- 172.21.15.146:0/1246313149 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f2e1c0026c0 con 0x7f2e2c1011c0 2024-09-11T07:57:01.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.601+0000 7f2e13fff640 1 -- 172.21.15.146:0/1246313149 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 239+0+0 (unknown 2620338839 0 0) 0x7f2e1c003a30 con 0x7f2e2c1011c0 2024-09-11T07:57:01.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.602+0000 7f2e13fff640 1 -- 172.21.15.146:0/1246313149 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 1600948874 0 0) 0x7f2e1c003c50 con 0x7f2e2c1011c0 2024-09-11T07:57:01.604 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.602+0000 7f2e3215e640 1 -- 172.21.15.146:0/1246313149 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e18005010 con 0x7f2e2c1011c0 2024-09-11T07:57:01.605 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.602+0000 7f2e13fff640 1 -- 172.21.15.146:0/1246313149 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (unknown 1709346042 0 0) 0x7f2e1c016b20 con 0x7f2e2c1011c0 2024-09-11T07:57:01.609 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.607+0000 7f2e13fff640 1 -- 172.21.15.146:0/1246313149 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f2e1c016d80 con 0x7f2e2c1011c0 2024-09-11T07:57:01.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.826+0000 7f2e3215e640 1 -- 172.21.15.146:0/1246313149 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2e180052c0 con 0x7f2e2c1011c0 2024-09-11T07:57:01.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.827+0000 7f2e13fff640 1 -- 172.21.15.146:0/1246313149 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+901 (unknown 3557084514 0 3497881446) 0x7f2e1c016780 con 0x7f2e2c1011c0 2024-09-11T07:57:01.829 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:57:01.829 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":2,"fsid":"2051892c-7013-11ef-bcea-c7b262605968","modified":"2024-09-11T07:56:56.583404Z","created":"2024-09-11T07:55:28.194357Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6789","nonce":0}]},"addr":"172.21.15.146:6789/0","public_addr":"172.21.15.146:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T07:57:01.829 INFO:teuthology.orchestra.run.smithi146.stderr:dumped monmap epoch 2 2024-09-11T07:57:01.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.829+0000 7f2e3215e640 1 -- 172.21.15.146:0/1246313149 >> v1:172.21.15.102:6800/2950771867 conn(0x7f2e1403a910 legacy=0x7f2e1403cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:01.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.829+0000 7f2e3215e640 1 -- 172.21.15.146:0/1246313149 >> v1:172.21.15.102:6789/0 conn(0x7f2e2c1011c0 legacy=0x7f2e2c111960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:01.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.829+0000 7f2e3215e640 1 -- 172.21.15.146:0/1246313149 shutdown_connections 2024-09-11T07:57:01.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.829+0000 7f2e3215e640 1 -- 172.21.15.146:0/1246313149 >> 172.21.15.146:0/1246313149 conn(0x7f2e2c071150 msgr2=0x7f2e2c064eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:57:01.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.829+0000 7f2e3215e640 1 -- 172.21.15.146:0/1246313149 shutdown_connections 2024-09-11T07:57:01.831 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:01.829+0000 7f2e3215e640 1 -- 172.21.15.146:0/1246313149 wait complete. 2024-09-11T07:57:01.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:01 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:57:01.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:01 smithi102 ceph-mon[29849]: mon.a calling monitor election 2024-09-11T07:57:01.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:01 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:57:01.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:01 smithi102 ceph-mon[29849]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:01.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:01 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:57:01.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:01 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:57:01.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:01 smithi102 ceph-mon[29849]: mon.b calling monitor election 2024-09-11T07:57:01.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:01 smithi102 ceph-mon[29849]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:01.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:01 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:57:01.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:01 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:57:01.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:01 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:57:01.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:01 smithi102 ceph-mon[29849]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T07:57:01.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:01 smithi102 ceph-mon[29849]: monmap e2: 2 mons at {a=v1:172.21.15.102:6789/0,b=v1:172.21.15.146:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:57:01.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:01 smithi102 ceph-mon[29849]: fsmap 2024-09-11T07:57:01.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:01 smithi102 ceph-mon[29849]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:57:01.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:01 smithi102 ceph-mon[29849]: mgrmap e13: y(active, since 38s) 2024-09-11T07:57:01.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:01 smithi102 ceph-mon[29849]: overall HEALTH_OK 2024-09-11T07:57:01.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:01 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:01.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:01 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:01.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:01 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:01.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:01 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:57:01.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:01 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:02.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:02 smithi102 ceph-mon[29849]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:02.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:02 smithi102 ceph-mon[29849]: Deploying daemon mon.c on smithi102 2024-09-11T07:57:02.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:02 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.146:0/1246313149' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:57:02.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:02 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:57:02.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:02 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:57:03.403 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T07:57:03.404 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T07:57:03.664 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:57:04.580 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 podman[41334]: 2024-09-11 07:57:04.469829629 +0000 UTC m=+0.137154216 container create 1a4e9f1b37da8a545c09af3ef44b6c0938f2428f9087e1179fbf3e8bd1777c55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:57:04.732 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:04 smithi146 ceph-mon[33158]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:05.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[29849]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:05.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 podman[41334]: 2024-09-11 07:57:04.714073401 +0000 UTC m=+0.381397982 container init 1a4e9f1b37da8a545c09af3ef44b6c0938f2428f9087e1179fbf3e8bd1777c55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-c, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-11T07:57:05.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 podman[41334]: 2024-09-11 07:57:04.719280522 +0000 UTC m=+0.386605106 container start 1a4e9f1b37da8a545c09af3ef44b6c0938f2428f9087e1179fbf3e8bd1777c55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:57:05.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T07:57:05.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable), process ceph-mon, pid 7 2024-09-11T07:57:05.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: pidfile_write: ignore empty --pid-file 2024-09-11T07:57:05.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: load: jerasure load: lrc 2024-09-11T07:57:05.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: RocksDB version: 6.15.5 2024-09-11T07:57:05.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-11T07:57:05.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Compile date Sep 10 2024 2024-09-11T07:57:05.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: DB SUMMARY 2024-09-11T07:57:05.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: DB Session ID: CMJU9Z4BV0XVY1NFVU8M 2024-09-11T07:57:05.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: CURRENT file: CURRENT 2024-09-11T07:57:05.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T07:57:05.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-11T07:57:05.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-11T07:57:05.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 566 ; 2024-09-11T07:57:05.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.error_if_exists: 0 2024-09-11T07:57:05.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.create_if_missing: 0 2024-09-11T07:57:05.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.paranoid_checks: 1 2024-09-11T07:57:05.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T07:57:05.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.env: 0x559044e66440 2024-09-11T07:57:05.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.fs: Posix File System 2024-09-11T07:57:05.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.info_log: 0x559046738e80 2024-09-11T07:57:05.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T07:57:05.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.statistics: (nil) 2024-09-11T07:57:05.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.use_fsync: 0 2024-09-11T07:57:05.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_log_file_size: 0 2024-09-11T07:57:05.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T07:57:05.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T07:57:05.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T07:57:05.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T07:57:05.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.allow_fallocate: 1 2024-09-11T07:57:05.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T07:57:05.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T07:57:05.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.use_direct_reads: 0 2024-09-11T07:57:05.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T07:57:05.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T07:57:05.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.db_log_dir: 2024-09-11T07:57:05.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2024-09-11T07:57:05.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T07:57:05.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T07:57:05.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T07:57:05.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T07:57:05.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T07:57:05.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T07:57:05.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T07:57:05.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T07:57:05.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.write_buffer_manager: 0x5590467f6840 2024-09-11T07:57:05.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T07:57:05.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-11T07:57:05.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T07:57:05.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T07:57:05.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T07:57:05.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T07:57:05.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T07:57:05.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T07:57:05.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T07:57:05.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.unordered_write: 0 2024-09-11T07:57:05.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T07:57:05.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T07:57:05.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T07:57:05.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T07:57:05.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.row_cache: None 2024-09-11T07:57:05.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.wal_filter: None 2024-09-11T07:57:05.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T07:57:05.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T07:57:05.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.preserve_deletes: 0 2024-09-11T07:57:05.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.two_write_queues: 0 2024-09-11T07:57:05.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T07:57:05.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.atomic_flush: 0 2024-09-11T07:57:05.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T07:57:05.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T07:57:05.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T07:57:05.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.log_readahead_size: 0 2024-09-11T07:57:05.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T07:57:05.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T07:57:05.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T07:57:05.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T07:57:05.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T07:57:05.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T07:57:05.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_background_jobs: 2 2024-09-11T07:57:05.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_background_compactions: -1 2024-09-11T07:57:05.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_subcompactions: 1 2024-09-11T07:57:05.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T07:57:05.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T07:57:05.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T07:57:05.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T07:57:05.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T07:57:05.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T07:57:05.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T07:57:05.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T07:57:05.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_open_files: -1 2024-09-11T07:57:05.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T07:57:05.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T07:57:05.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T07:57:05.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T07:57:05.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_background_flushes: -1 2024-09-11T07:57:05.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Compression algorithms supported: 2024-09-11T07:57:05.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: kZSTD supported: 0 2024-09-11T07:57:05.155 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: kXpressCompression supported: 0 2024-09-11T07:57:05.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: kBZip2Compression supported: 0 2024-09-11T07:57:05.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T07:57:05.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: kLZ4Compression supported: 1 2024-09-11T07:57:05.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: kZlibCompression supported: 1 2024-09-11T07:57:05.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T07:57:05.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: kSnappyCompression supported: 1 2024-09-11T07:57:05.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T07:57:05.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 2024-09-11T07:57:05.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-11T07:57:05.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T07:57:05.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.merge_operator: 2024-09-11T07:57:05.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.compaction_filter: None 2024-09-11T07:57:05.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T07:57:05.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T07:57:05.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T07:57:05.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T07:57:05.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5590466dfa38) 2024-09-11T07:57:05.157 INFO:journalctl@ceph.mon.c.smithi102.stdout: cache_index_and_filter_blocks: 1 2024-09-11T07:57:05.157 INFO:journalctl@ceph.mon.c.smithi102.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T07:57:05.157 INFO:journalctl@ceph.mon.c.smithi102.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T07:57:05.157 INFO:journalctl@ceph.mon.c.smithi102.stdout: pin_top_level_index_and_filter: 1 2024-09-11T07:57:05.157 INFO:journalctl@ceph.mon.c.smithi102.stdout: index_type: 0 2024-09-11T07:57:05.157 INFO:journalctl@ceph.mon.c.smithi102.stdout: data_block_index_type: 0 2024-09-11T07:57:05.157 INFO:journalctl@ceph.mon.c.smithi102.stdout: index_shortening: 1 2024-09-11T07:57:05.157 INFO:journalctl@ceph.mon.c.smithi102.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T07:57:05.157 INFO:journalctl@ceph.mon.c.smithi102.stdout: hash_index_allow_collision: 1 2024-09-11T07:57:05.158 INFO:journalctl@ceph.mon.c.smithi102.stdout: checksum: 1 2024-09-11T07:57:05.158 INFO:journalctl@ceph.mon.c.smithi102.stdout: no_block_cache: 0 2024-09-11T07:57:05.158 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_cache: 0x5590467151f0 2024-09-11T07:57:05.158 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_cache_name: BinnedLRUCache 2024-09-11T07:57:05.158 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_cache_options: 2024-09-11T07:57:05.158 INFO:journalctl@ceph.mon.c.smithi102.stdout: capacity : 536870912 2024-09-11T07:57:05.158 INFO:journalctl@ceph.mon.c.smithi102.stdout: num_shard_bits : 4 2024-09-11T07:57:05.158 INFO:journalctl@ceph.mon.c.smithi102.stdout: strict_capacity_limit : 0 2024-09-11T07:57:05.158 INFO:journalctl@ceph.mon.c.smithi102.stdout: high_pri_pool_ratio: 0.000 2024-09-11T07:57:05.158 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_cache_compressed: (nil) 2024-09-11T07:57:05.158 INFO:journalctl@ceph.mon.c.smithi102.stdout: persistent_cache: (nil) 2024-09-11T07:57:05.158 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_size: 4096 2024-09-11T07:57:05.158 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_size_deviation: 10 2024-09-11T07:57:05.159 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_restart_interval: 16 2024-09-11T07:57:05.159 INFO:journalctl@ceph.mon.c.smithi102.stdout: index_block_restart_interval: 1 2024-09-11T07:57:05.159 INFO:journalctl@ceph.mon.c.smithi102.stdout: metadata_block_size: 4096 2024-09-11T07:57:05.159 INFO:journalctl@ceph.mon.c.smithi102.stdout: partition_filters: 0 2024-09-11T07:57:05.159 INFO:journalctl@ceph.mon.c.smithi102.stdout: use_delta_encoding: 1 2024-09-11T07:57:05.159 INFO:journalctl@ceph.mon.c.smithi102.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-11T07:57:05.159 INFO:journalctl@ceph.mon.c.smithi102.stdout: whole_key_filtering: 1 2024-09-11T07:57:05.159 INFO:journalctl@ceph.mon.c.smithi102.stdout: verify_compression: 0 2024-09-11T07:57:05.159 INFO:journalctl@ceph.mon.c.smithi102.stdout: read_amp_bytes_per_bit: 0 2024-09-11T07:57:05.159 INFO:journalctl@ceph.mon.c.smithi102.stdout: format_version: 4 2024-09-11T07:57:05.159 INFO:journalctl@ceph.mon.c.smithi102.stdout: enable_index_compression: 1 2024-09-11T07:57:05.159 INFO:journalctl@ceph.mon.c.smithi102.stdout: block_align: 0 2024-09-11T07:57:05.159 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T07:57:05.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T07:57:05.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.compression: NoCompression 2024-09-11T07:57:05.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T07:57:05.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T07:57:05.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T07:57:05.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.num_levels: 7 2024-09-11T07:57:05.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T07:57:05.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T07:57:05.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T07:57:05.160 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T07:57:05.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T07:57:05.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T07:57:05.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T07:57:05.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T07:57:05.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T07:57:05.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T07:57:05.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T07:57:05.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T07:57:05.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T07:57:05.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T07:57:05.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T07:57:05.161 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T07:57:05.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T07:57:05.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T07:57:05.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T07:57:05.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T07:57:05.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T07:57:05.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T07:57:05.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T07:57:05.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T07:57:05.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T07:57:05.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T07:57:05.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T07:57:05.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T07:57:05.162 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T07:57:05.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T07:57:05.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T07:57:05.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T07:57:05.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T07:57:05.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T07:57:05.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.arena_block_size: 4194304 2024-09-11T07:57:05.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T07:57:05.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T07:57:05.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-11T07:57:05.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T07:57:05.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T07:57:05.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T07:57:05.163 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T07:57:05.164 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T07:57:05.164 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T07:57:05.164 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T07:57:05.164 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T07:57:05.164 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T07:57:05.164 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T07:57:05.164 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T07:57:05.164 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T07:57:05.164 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.inplace_update_support: 0 2024-09-11T07:57:05.164 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T07:57:05.164 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T07:57:05.164 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T07:57:05.165 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T07:57:05.165 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.bloom_locality: 0 2024-09-11T07:57:05.165 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.max_successive_merges: 0 2024-09-11T07:57:05.165 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T07:57:05.165 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T07:57:05.165 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T07:57:05.165 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T07:57:05.165 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.ttl: 2592000 2024-09-11T07:57:05.165 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T07:57:05.165 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.enable_blob_files: false 2024-09-11T07:57:05.165 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.min_blob_size: 0 2024-09-11T07:57:05.165 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T07:57:05.165 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T07:57:05.165 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T07:57:05.165 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T07:57:05.166 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-11T07:57:05.166 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-11T07:57:05.166 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-11T07:57:05.166 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041424742925, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-11T07:57:05.166 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-11T07:57:05.166 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-11T07:57:05.166 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041424743215, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1626, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 578, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 456, "raw_average_value_size": 91, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1726041424, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "9ae7fd15-bd56-475a-ad69-6a0a05fe73b5", "db_session_id": "CMJU9Z4BV0XVY1NFVU8M"}} 2024-09-11T07:57:05.166 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-11T07:57:05.166 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726041424743527, "job": 1, "event": "recovery_finished"} 2024-09-11T07:57:05.166 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T07:57:05.166 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x559046748700 2024-09-11T07:57:05.166 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: DB pointer 0x559046718000 2024-09-11T07:57:05.167 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-11T07:57:05.167 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-11T07:57:05.167 INFO:journalctl@ceph.mon.c.smithi102.stdout: ** DB Stats ** 2024-09-11T07:57:05.167 INFO:journalctl@ceph.mon.c.smithi102.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:57:05.167 INFO:journalctl@ceph.mon.c.smithi102.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T07:57:05.167 INFO:journalctl@ceph.mon.c.smithi102.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T07:57:05.167 INFO:journalctl@ceph.mon.c.smithi102.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T07:57:05.167 INFO:journalctl@ceph.mon.c.smithi102.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T07:57:05.167 INFO:journalctl@ceph.mon.c.smithi102.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-11T07:57:05.167 INFO:journalctl@ceph.mon.c.smithi102.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T07:57:05.167 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-09-11T07:57:05.167 INFO:journalctl@ceph.mon.c.smithi102.stdout: ** Compaction Stats [default] ** 2024-09-11T07:57:05.167 INFO:journalctl@ceph.mon.c.smithi102.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:57:05.167 INFO:journalctl@ceph.mon.c.smithi102.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:57:05.167 INFO:journalctl@ceph.mon.c.smithi102.stdout: L0 1/0 1.59 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-11T07:57:05.168 INFO:journalctl@ceph.mon.c.smithi102.stdout: Sum 1/0 1.59 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-11T07:57:05.168 INFO:journalctl@ceph.mon.c.smithi102.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-11T07:57:05.168 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-09-11T07:57:05.168 INFO:journalctl@ceph.mon.c.smithi102.stdout: ** Compaction Stats [default] ** 2024-09-11T07:57:05.168 INFO:journalctl@ceph.mon.c.smithi102.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:57:05.168 INFO:journalctl@ceph.mon.c.smithi102.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:57:05.168 INFO:journalctl@ceph.mon.c.smithi102.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-11T07:57:05.168 INFO:journalctl@ceph.mon.c.smithi102.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:57:05.168 INFO:journalctl@ceph.mon.c.smithi102.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T07:57:05.168 INFO:journalctl@ceph.mon.c.smithi102.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T07:57:05.168 INFO:journalctl@ceph.mon.c.smithi102.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T07:57:05.168 INFO:journalctl@ceph.mon.c.smithi102.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T07:57:05.168 INFO:journalctl@ceph.mon.c.smithi102.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T07:57:05.169 INFO:journalctl@ceph.mon.c.smithi102.stdout: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:57:05.169 INFO:journalctl@ceph.mon.c.smithi102.stdout: Interval compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:57:05.169 INFO:journalctl@ceph.mon.c.smithi102.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T07:57:05.169 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-09-11T07:57:05.169 INFO:journalctl@ceph.mon.c.smithi102.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T07:57:05.169 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-09-11T07:57:05.169 INFO:journalctl@ceph.mon.c.smithi102.stdout: ** Compaction Stats [default] ** 2024-09-11T07:57:05.169 INFO:journalctl@ceph.mon.c.smithi102.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:57:05.169 INFO:journalctl@ceph.mon.c.smithi102.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:57:05.169 INFO:journalctl@ceph.mon.c.smithi102.stdout: L0 1/0 1.59 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-11T07:57:05.169 INFO:journalctl@ceph.mon.c.smithi102.stdout: Sum 1/0 1.59 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-11T07:57:05.169 INFO:journalctl@ceph.mon.c.smithi102.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-11T07:57:05.169 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-09-11T07:57:05.169 INFO:journalctl@ceph.mon.c.smithi102.stdout: ** Compaction Stats [default] ** 2024-09-11T07:57:05.170 INFO:journalctl@ceph.mon.c.smithi102.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:57:05.170 INFO:journalctl@ceph.mon.c.smithi102.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:57:05.170 INFO:journalctl@ceph.mon.c.smithi102.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.6 0.00 0.00 1 0.000 0 0 2024-09-11T07:57:05.170 INFO:journalctl@ceph.mon.c.smithi102.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:57:05.170 INFO:journalctl@ceph.mon.c.smithi102.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T07:57:05.170 INFO:journalctl@ceph.mon.c.smithi102.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T07:57:05.170 INFO:journalctl@ceph.mon.c.smithi102.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T07:57:05.170 INFO:journalctl@ceph.mon.c.smithi102.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T07:57:05.170 INFO:journalctl@ceph.mon.c.smithi102.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T07:57:05.170 INFO:journalctl@ceph.mon.c.smithi102.stdout: Cumulative compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:57:05.170 INFO:journalctl@ceph.mon.c.smithi102.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:57:05.170 INFO:journalctl@ceph.mon.c.smithi102.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T07:57:05.170 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-09-11T07:57:05.171 INFO:journalctl@ceph.mon.c.smithi102.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T07:57:05.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-11T07:57:05.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: using public_addrv v1:172.21.15.102:6790/0 2024-09-11T07:57:05.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: starting mon.c rank -1 at public addrs v1:172.21.15.102:6790/0 at bind addrs v1:172.21.15.102:6790/0 mon_data /var/lib/ceph/mon/ceph-c fsid 2051892c-7013-11ef-bcea-c7b262605968 2024-09-11T07:57:05.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: mon.c@-1(???) e0 preinit fsid 2051892c-7013-11ef-bcea-c7b262605968 2024-09-11T07:57:05.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: mon.c@-1(synchronizing).mds e1 new map 2024-09-11T07:57:05.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-11T07:57:05.171 INFO:journalctl@ceph.mon.c.smithi102.stdout: e1 2024-09-11T07:57:05.171 INFO:journalctl@ceph.mon.c.smithi102.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T07:57:05.171 INFO:journalctl@ceph.mon.c.smithi102.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-11T07:57:05.171 INFO:journalctl@ceph.mon.c.smithi102.stdout: legacy client fscid: -1 2024-09-11T07:57:05.171 INFO:journalctl@ceph.mon.c.smithi102.stdout: 2024-09-11T07:57:05.171 INFO:journalctl@ceph.mon.c.smithi102.stdout: No filesystems configured 2024-09-11T07:57:05.171 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-11T07:57:05.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-11T07:57:05.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-11T07:57:05.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-11T07:57:05.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-11T07:57:05.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-11T07:57:05.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T07:57:05.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:57:05.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:57:05.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:57:05.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: Updating smithi146:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:57:05.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: Updating smithi146:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:57:05.172 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: Updating smithi146:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:57:05.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3700124607' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T07:57:05.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:57:05.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:05.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:05.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: from='client.14182 v1:172.21.15.146:0/1903928884' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi102:[v1:172.21.15.102:6789]=a;smithi102:[v1:172.21.15.102:6790]=c;smithi146:[v1:172.21.15.146:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:05.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: Saving service mon spec with placement smithi102:[v1:172.21.15.102:6789]=a;smithi102:[v1:172.21.15.102:6790]=c;smithi146:[v1:172.21.15.146:6789]=b;count:3 2024-09-11T07:57:05.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:05.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:05.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:05.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:05.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:05.173 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:57:05.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:05.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: Deploying daemon mon.b on smithi146 2024-09-11T07:57:05.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:05.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:57:05.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: mon.a calling monitor election 2024-09-11T07:57:05.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:57:05.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:05.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:57:05.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:57:05.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: mon.b calling monitor election 2024-09-11T07:57:05.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:05.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:57:05.174 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:57:05.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:57:05.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T07:57:05.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: monmap e2: 2 mons at {a=v1:172.21.15.102:6789/0,b=v1:172.21.15.146:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:57:05.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: fsmap 2024-09-11T07:57:05.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:57:05.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: mgrmap e13: y(active, since 38s) 2024-09-11T07:57:05.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: overall HEALTH_OK 2024-09-11T07:57:05.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:05.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:05.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:05.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:57:05.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:05.175 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:05.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: Deploying daemon mon.c on smithi102 2024-09-11T07:57:05.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.146:0/1246313149' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:57:05.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:57:05.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:05.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 ceph-mon[41371]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-11T07:57:05.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 bash[41334]: 1a4e9f1b37da8a545c09af3ef44b6c0938f2428f9087e1179fbf3e8bd1777c55 2024-09-11T07:57:05.176 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:04 smithi102 systemd[1]: Started Ceph mon.c for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T07:57:09.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.774+0000 7f8423f5b640 1 -- 172.21.15.146:0/1078912336 >> v1:172.21.15.102:6789/0 conn(0x7f8400003520 legacy=0x7f84000059a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:09.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.774+0000 7f8423f5b640 1 -- 172.21.15.146:0/1078912336 shutdown_connections 2024-09-11T07:57:09.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.774+0000 7f8423f5b640 1 -- 172.21.15.146:0/1078912336 >> 172.21.15.146:0/1078912336 conn(0x7f841c0717c0 msgr2=0x7f841c071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:57:09.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.774+0000 7f8423f5b640 1 -- 172.21.15.146:0/1078912336 shutdown_connections 2024-09-11T07:57:09.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.774+0000 7f8423f5b640 1 -- 172.21.15.146:0/1078912336 wait complete. 2024-09-11T07:57:09.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.774+0000 7f8423f5b640 1 Processor -- start 2024-09-11T07:57:09.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.774+0000 7f8423f5b640 1 -- start start 2024-09-11T07:57:09.777 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.775+0000 7f8423f5b640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f841c11bb50 con 0x7f841c112070 2024-09-11T07:57:09.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.775+0000 7f8423f5b640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f841c11cd10 con 0x7f841c117e90 2024-09-11T07:57:09.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.775+0000 7f8423f5b640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f841c11df10 con 0x7f8400003520 2024-09-11T07:57:09.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.775+0000 7f84224f8640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f841c117e90 0x7f841c11a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:48942/0 (socket says 172.21.15.146:48942) 2024-09-11T07:57:09.778 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.775+0000 7f84224f8640 1 -- 172.21.15.146:0/1471029384 learned_addr learned my addr 172.21.15.146:0/1471029384 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-11T07:57:09.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.776+0000 7f840e7fc640 1 -- 172.21.15.146:0/1471029384 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302216133 0 0) 0x7f841c11bb50 con 0x7f841c112070 2024-09-11T07:57:09.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.776+0000 7f840e7fc640 1 -- 172.21.15.146:0/1471029384 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8408003b80 con 0x7f841c112070 2024-09-11T07:57:09.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.776+0000 7f840e7fc640 1 -- 172.21.15.146:0/1471029384 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3216249399 0 0) 0x7f8408003b80 con 0x7f841c112070 2024-09-11T07:57:09.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.776+0000 7f840e7fc640 1 -- 172.21.15.146:0/1471029384 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f841c11bb50 con 0x7f841c112070 2024-09-11T07:57:09.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.776+0000 7f840e7fc640 1 -- 172.21.15.146:0/1471029384 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f8418001840 con 0x7f841c112070 2024-09-11T07:57:09.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.776+0000 7f840e7fc640 1 -- 172.21.15.146:0/1471029384 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 665794502 0 0) 0x7f841c11bb50 con 0x7f841c112070 2024-09-11T07:57:09.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.777+0000 7f840e7fc640 1 -- 172.21.15.146:0/1471029384 >> v1:172.21.15.102:6790/0 conn(0x7f8400003520 legacy=0x7f841c111960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:09.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.777+0000 7f840e7fc640 1 -- 172.21.15.146:0/1471029384 >> v1:172.21.15.146:6789/0 conn(0x7f841c117e90 legacy=0x7f841c11a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:09.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.777+0000 7f840e7fc640 1 -- 172.21.15.146:0/1471029384 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841c11f110 con 0x7f841c112070 2024-09-11T07:57:09.779 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.777+0000 7f8423f5b640 1 -- 172.21.15.146:0/1471029384 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f841c11cf00 con 0x7f841c112070 2024-09-11T07:57:09.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.777+0000 7f8423f5b640 1 -- 172.21.15.146:0/1471029384 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f841c11d390 con 0x7f841c112070 2024-09-11T07:57:09.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.777+0000 7f840e7fc640 1 -- 172.21.15.146:0/1471029384 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f8418002350 con 0x7f841c112070 2024-09-11T07:57:09.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.777+0000 7f840e7fc640 1 -- 172.21.15.146:0/1471029384 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f8418003120 con 0x7f841c112070 2024-09-11T07:57:09.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.778+0000 7f840e7fc640 1 -- 172.21.15.146:0/1471029384 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 1600948874 0 0) 0x7f841800e460 con 0x7f841c112070 2024-09-11T07:57:09.780 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.778+0000 7f840e7fc640 1 -- 172.21.15.146:0/1471029384 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (unknown 1709346042 0 0) 0x7f8418044df0 con 0x7f841c112070 2024-09-11T07:57:09.781 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.778+0000 7f8423f5b640 1 -- 172.21.15.146:0/1471029384 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8400006a50 con 0x7f841c112070 2024-09-11T07:57:09.784 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:09.783+0000 7f840e7fc640 1 -- 172.21.15.146:0/1471029384 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f841800fd50 con 0x7f841c112070 2024-09-11T07:57:10.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:10.013+0000 7f8423f5b640 1 -- 172.21.15.146:0/1471029384 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8400006d00 con 0x7f841c112070 2024-09-11T07:57:10.015 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:10.014+0000 7f840e7fc640 1 -- 172.21.15.146:0/1471029384 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1117 (unknown 409899127 0 3002513506) 0x7f84180137c0 con 0x7f841c112070 2024-09-11T07:57:10.015 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-11T07:57:10.016 INFO:teuthology.orchestra.run.smithi146.stdout:{"epoch":3,"fsid":"2051892c-7013-11ef-bcea-c7b262605968","modified":"2024-09-11T07:57:04.763983Z","created":"2024-09-11T07:55:28.194357Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6789","nonce":0}]},"addr":"172.21.15.146:6789/0","public_addr":"172.21.15.146:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6790","nonce":0}]},"addr":"172.21.15.102:6790/0","public_addr":"172.21.15.102:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T07:57:10.016 INFO:teuthology.orchestra.run.smithi146.stderr:dumped monmap epoch 3 2024-09-11T07:57:10.017 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:10.016+0000 7f8423f5b640 1 -- 172.21.15.146:0/1471029384 >> v1:172.21.15.102:6800/2950771867 conn(0x7f840803a710 legacy=0x7f840803cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:10.017 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:10.016+0000 7f8423f5b640 1 -- 172.21.15.146:0/1471029384 >> v1:172.21.15.102:6789/0 conn(0x7f841c112070 legacy=0x7f841c116770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:10.017 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:10.016+0000 7f8423f5b640 1 -- 172.21.15.146:0/1471029384 shutdown_connections 2024-09-11T07:57:10.017 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:10.016+0000 7f8423f5b640 1 -- 172.21.15.146:0/1471029384 >> 172.21.15.146:0/1471029384 conn(0x7f841c0717c0 msgr2=0x7f841c1017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:57:10.018 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:10.016+0000 7f8423f5b640 1 -- 172.21.15.146:0/1471029384 shutdown_connections 2024-09-11T07:57:10.018 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:10.016+0000 7f8423f5b640 1 -- 172.21.15.146:0/1471029384 wait complete. 2024-09-11T07:57:10.027 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:57:10.027 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: mon.a calling monitor election 2024-09-11T07:57:10.027 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: mon.b calling monitor election 2024-09-11T07:57:10.027 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:57:10.027 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:10.027 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:10.027 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:10.027 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:10.027 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:10.027 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:10.027 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:10.028 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:10.028 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T07:57:10.028 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: monmap e3: 3 mons at {a=v1:172.21.15.102:6789/0,b=v1:172.21.15.146:6789/0,c=v1:172.21.15.102:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:57:10.028 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: fsmap 2024-09-11T07:57:10.028 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:57:10.028 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: mgrmap e13: y(active, since 46s) 2024-09-11T07:57:10.028 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-11T07:57:10.028 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-11T07:57:10.028 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-11T07:57:10.028 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: mon.c (rank 2) addr v1:172.21.15.102:6790/0 is down (out of quorum) 2024-09-11T07:57:10.028 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:10.028 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:10.028 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:10.029 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:09 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:57:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: mon.a calling monitor election 2024-09-11T07:57:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: mon.b calling monitor election 2024-09-11T07:57:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:57:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T07:57:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: monmap e3: 3 mons at {a=v1:172.21.15.102:6789/0,b=v1:172.21.15.146:6789/0,c=v1:172.21.15.102:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:57:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: fsmap 2024-09-11T07:57:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:57:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: mgrmap e13: y(active, since 46s) 2024-09-11T07:57:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-11T07:57:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-11T07:57:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-11T07:57:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: mon.c (rank 2) addr v1:172.21.15.102:6790/0 is down (out of quorum) 2024-09-11T07:57:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:10.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:09 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:10.645 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-11T07:57:10.645 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-11T07:57:10.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:10 smithi146 ceph-mon[33158]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:10.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:10 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:10.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.146:0/1471029384' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:57:10.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:10 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:10.973 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:57:11.096 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:10 smithi102 ceph-mon[29849]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:11.096 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:10 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:11.096 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.146:0/1471029384' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:57:11.096 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:10 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:12.126 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:57:12.127 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: mon.a calling monitor election 2024-09-11T07:57:12.127 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: mon.b calling monitor election 2024-09-11T07:57:12.127 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:57:12.127 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:12.127 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:12.127 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:12.127 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:12.127 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:12.127 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:12.127 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:12.127 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:12.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T07:57:12.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: monmap e3: 3 mons at {a=v1:172.21.15.102:6789/0,b=v1:172.21.15.146:6789/0,c=v1:172.21.15.102:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:57:12.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: fsmap 2024-09-11T07:57:12.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:57:12.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: mgrmap e13: y(active, since 46s) 2024-09-11T07:57:12.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-11T07:57:12.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-11T07:57:12.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-11T07:57:12.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: mon.c (rank 2) addr v1:172.21.15.102:6790/0 is down (out of quorum) 2024-09-11T07:57:12.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:12.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:12.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:12.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:12.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:12.129 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:12.129 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.146:0/1471029384' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:57:12.129 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:11 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:12.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.653+0000 7fddc9b30640 1 -- 172.21.15.102:0/2522293169 >> v1:172.21.15.102:6789/0 conn(0x7fddc4102120 legacy=0x7fddc41024f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:12.679 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.654+0000 7fddc9b30640 1 -- 172.21.15.102:0/2522293169 shutdown_connections 2024-09-11T07:57:12.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.654+0000 7fddc9b30640 1 -- 172.21.15.102:0/2522293169 >> 172.21.15.102:0/2522293169 conn(0x7fddc40f7cd0 msgr2=0x7fddc40fa0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:57:12.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.654+0000 7fddc9b30640 1 -- 172.21.15.102:0/2522293169 shutdown_connections 2024-09-11T07:57:12.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.654+0000 7fddc9b30640 1 -- 172.21.15.102:0/2522293169 wait complete. 2024-09-11T07:57:12.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.654+0000 7fddc9b30640 1 Processor -- start 2024-09-11T07:57:12.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.654+0000 7fddc9b30640 1 -- start start 2024-09-11T07:57:12.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.655+0000 7fddc9b30640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fddc411bb00 con 0x7fddc4112020 2024-09-11T07:57:12.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.655+0000 7fddc9b30640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fddc411ccc0 con 0x7fddc4117e40 2024-09-11T07:57:12.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.655+0000 7fddc9b30640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fddc411dec0 con 0x7fddc4102120 2024-09-11T07:57:12.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.655+0000 7fddc2ffd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fddc4112020 0x7fddc4116720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:43850/0 (socket says 172.21.15.102:43850) 2024-09-11T07:57:12.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.655+0000 7fddc2ffd640 1 -- 172.21.15.102:0/2263666564 learned_addr learned my addr 172.21.15.102:0/2263666564 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:57:12.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.655+0000 7fddbbfff640 1 -- 172.21.15.102:0/2263666564 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1586459319 0 0) 0x7fddc411bb00 con 0x7fddc4112020 2024-09-11T07:57:12.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.655+0000 7fddbbfff640 1 -- 172.21.15.102:0/2263666564 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddb00034e0 con 0x7fddc4112020 2024-09-11T07:57:12.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.655+0000 7fddbbfff640 1 -- 172.21.15.102:0/2263666564 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1788390705 0 0) 0x7fddc411dec0 con 0x7fddc4102120 2024-09-11T07:57:12.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.655+0000 7fddbbfff640 1 -- 172.21.15.102:0/2263666564 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddc411bb00 con 0x7fddc4102120 2024-09-11T07:57:12.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.655+0000 7fddbbfff640 1 -- 172.21.15.102:0/2263666564 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1513157524 0 0) 0x7fddb00034e0 con 0x7fddc4112020 2024-09-11T07:57:12.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.655+0000 7fddbbfff640 1 -- 172.21.15.102:0/2263666564 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fddc411dec0 con 0x7fddc4112020 2024-09-11T07:57:12.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.655+0000 7fddbbfff640 1 -- 172.21.15.102:0/2263666564 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fddac001770 con 0x7fddc4112020 2024-09-11T07:57:12.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.655+0000 7fddbbfff640 1 -- 172.21.15.102:0/2263666564 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 46739911 0 0) 0x7fddc411ccc0 con 0x7fddc4117e40 2024-09-11T07:57:12.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.655+0000 7fddbbfff640 1 -- 172.21.15.102:0/2263666564 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddb00034e0 con 0x7fddc4117e40 2024-09-11T07:57:12.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.655+0000 7fddbbfff640 1 -- 172.21.15.102:0/2263666564 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2640977412 0 0) 0x7fddc411bb00 con 0x7fddc4102120 2024-09-11T07:57:12.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.655+0000 7fddbbfff640 1 -- 172.21.15.102:0/2263666564 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fddc411ccc0 con 0x7fddc4102120 2024-09-11T07:57:12.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.655+0000 7fddbbfff640 1 -- 172.21.15.102:0/2263666564 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fddb40023d0 con 0x7fddc4102120 2024-09-11T07:57:12.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.655+0000 7fddbbfff640 1 -- 172.21.15.102:0/2263666564 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 47543656 0 0) 0x7fddc411dec0 con 0x7fddc4112020 2024-09-11T07:57:12.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.655+0000 7fddbbfff640 1 -- 172.21.15.102:0/2263666564 >> v1:172.21.15.102:6790/0 conn(0x7fddc4102120 legacy=0x7fddc4111910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:12.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.655+0000 7fddbbfff640 1 -- 172.21.15.102:0/2263666564 >> v1:172.21.15.146:6789/0 conn(0x7fddc4117e40 legacy=0x7fddc411a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:12.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.655+0000 7fddbbfff640 1 -- 172.21.15.102:0/2263666564 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc411f0c0 con 0x7fddc4112020 2024-09-11T07:57:12.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.655+0000 7fddc9b30640 1 -- 172.21.15.102:0/2263666564 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc411bcf0 con 0x7fddc4112020 2024-09-11T07:57:12.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.655+0000 7fddc9b30640 1 -- 172.21.15.102:0/2263666564 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc411c1c0 con 0x7fddc4112020 2024-09-11T07:57:12.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.656+0000 7fddbbfff640 1 -- 172.21.15.102:0/2263666564 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7fddac001d60 con 0x7fddc4112020 2024-09-11T07:57:12.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.656+0000 7fddbbfff640 1 -- 172.21.15.102:0/2263666564 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fddac005ab0 con 0x7fddc4112020 2024-09-11T07:57:12.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.656+0000 7fddbbfff640 1 -- 172.21.15.102:0/2263666564 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 1600948874 0 0) 0x7fddac010df0 con 0x7fddc4112020 2024-09-11T07:57:12.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.656+0000 7fddc9b30640 1 -- 172.21.15.102:0/2263666564 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdda4005010 con 0x7fddc4112020 2024-09-11T07:57:12.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.657+0000 7fddbbfff640 1 -- 172.21.15.102:0/2263666564 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (unknown 1709346042 0 0) 0x7fddac047510 con 0x7fddc4112020 2024-09-11T07:57:12.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.661+0000 7fddbbfff640 1 -- 172.21.15.102:0/2263666564 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fddac012600 con 0x7fddc4112020 2024-09-11T07:57:12.683 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: mon.c calling monitor election 2024-09-11T07:57:12.683 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:12.683 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: Updating smithi102:/etc/ceph/ceph.conf 2024-09-11T07:57:12.683 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: Updating smithi146:/etc/ceph/ceph.conf 2024-09-11T07:57:12.683 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: Updating smithi146:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:57:12.683 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: Updating smithi102:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:57:12.683 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: mon.c calling monitor election 2024-09-11T07:57:12.683 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: mon.b calling monitor election 2024-09-11T07:57:12.683 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: mon.a calling monitor election 2024-09-11T07:57:12.683 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T07:57:12.683 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: monmap e3: 3 mons at {a=v1:172.21.15.102:6789/0,b=v1:172.21.15.146:6789/0,c=v1:172.21.15.102:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:57:12.683 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: fsmap 2024-09-11T07:57:12.684 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:57:12.684 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: mgrmap e13: y(active, since 48s) 2024-09-11T07:57:12.684 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-11T07:57:12.684 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: Cluster is now healthy 2024-09-11T07:57:12.684 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: overall HEALTH_OK 2024-09-11T07:57:12.684 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:12.684 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:12.684 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:12.684 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:12.684 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:12.684 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:57:12.684 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:57:12.684 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:12.685 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: mon.c calling monitor election 2024-09-11T07:57:12.685 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:12.685 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: Updating smithi102:/etc/ceph/ceph.conf 2024-09-11T07:57:12.685 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: Updating smithi146:/etc/ceph/ceph.conf 2024-09-11T07:57:12.685 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: Updating smithi146:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:57:12.685 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: Updating smithi102:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:57:12.685 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: mon.c calling monitor election 2024-09-11T07:57:12.685 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: mon.b calling monitor election 2024-09-11T07:57:12.685 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: mon.a calling monitor election 2024-09-11T07:57:12.685 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T07:57:12.685 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: monmap e3: 3 mons at {a=v1:172.21.15.102:6789/0,b=v1:172.21.15.146:6789/0,c=v1:172.21.15.102:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:57:12.685 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: fsmap 2024-09-11T07:57:12.686 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:57:12.686 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: mgrmap e13: y(active, since 48s) 2024-09-11T07:57:12.686 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-11T07:57:12.686 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: Cluster is now healthy 2024-09-11T07:57:12.686 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: overall HEALTH_OK 2024-09-11T07:57:12.686 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:12.686 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:12.686 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:12.686 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:12.686 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:12.686 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:57:12.686 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:57:12.686 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:12 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:12.821 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.820+0000 7fddc9b30640 1 -- 172.21.15.102:0/2263666564 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fdda40052c0 con 0x7fddc4112020 2024-09-11T07:57:12.822 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.820+0000 7fddbbfff640 1 -- 172.21.15.102:0/2263666564 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+193 (unknown 2189217731 0 2407318815) 0x7fddac015ff0 con 0x7fddc4112020 2024-09-11T07:57:12.822 INFO:teuthology.orchestra.run.smithi102.stdout:# minimal ceph.conf for 2051892c-7013-11ef-bcea-c7b262605968 2024-09-11T07:57:12.822 INFO:teuthology.orchestra.run.smithi102.stdout:[global] 2024-09-11T07:57:12.822 INFO:teuthology.orchestra.run.smithi102.stdout: fsid = 2051892c-7013-11ef-bcea-c7b262605968 2024-09-11T07:57:12.822 INFO:teuthology.orchestra.run.smithi102.stdout: mon_host = 172.21.15.102:6789/0 172.21.15.146:6789/0 v1:172.21.15.102:6790/0 2024-09-11T07:57:12.823 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.822+0000 7fddc9b30640 1 -- 172.21.15.102:0/2263666564 >> v1:172.21.15.102:6800/2950771867 conn(0x7fddb003aaa0 legacy=0x7fddb003cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:12.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.822+0000 7fddc9b30640 1 -- 172.21.15.102:0/2263666564 >> v1:172.21.15.102:6789/0 conn(0x7fddc4112020 legacy=0x7fddc4116720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:12.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.822+0000 7fddc9b30640 1 -- 172.21.15.102:0/2263666564 shutdown_connections 2024-09-11T07:57:12.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.822+0000 7fddc9b30640 1 -- 172.21.15.102:0/2263666564 >> 172.21.15.102:0/2263666564 conn(0x7fddc40f7cd0 msgr2=0x7fddc40fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:57:12.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.822+0000 7fddc9b30640 1 -- 172.21.15.102:0/2263666564 shutdown_connections 2024-09-11T07:57:12.824 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:12.823+0000 7fddc9b30640 1 -- 172.21.15.102:0/2263666564 wait complete. 2024-09-11T07:57:12.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: mon.c calling monitor election 2024-09-11T07:57:12.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:12.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: Updating smithi102:/etc/ceph/ceph.conf 2024-09-11T07:57:12.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: Updating smithi146:/etc/ceph/ceph.conf 2024-09-11T07:57:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: Updating smithi146:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:57:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: Updating smithi102:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:57:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: mon.c calling monitor election 2024-09-11T07:57:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: mon.b calling monitor election 2024-09-11T07:57:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: mon.a calling monitor election 2024-09-11T07:57:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T07:57:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: monmap e3: 3 mons at {a=v1:172.21.15.102:6789/0,b=v1:172.21.15.146:6789/0,c=v1:172.21.15.102:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:57:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: fsmap 2024-09-11T07:57:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:57:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: mgrmap e13: y(active, since 48s) 2024-09-11T07:57:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-11T07:57:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: Cluster is now healthy 2024-09-11T07:57:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: overall HEALTH_OK 2024-09-11T07:57:12.936 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:12.936 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:12.936 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:12.936 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:12.936 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:12.936 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:57:12.936 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:57:12.936 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:12 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:13.536 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-11T07:57:13.536 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-11T07:57:13.536 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T07:57:13.563 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-11T07:57:13.563 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:57:13.632 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-11T07:57:13.632 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T07:57:13.666 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-11T07:57:13.666 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:57:13.736 INFO:tasks.cephadm:Adding mgr.y on smithi102 2024-09-11T07:57:13.736 INFO:tasks.cephadm:Adding mgr.x on smithi146 2024-09-11T07:57:13.736 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph orch apply mgr '2;smithi102=y;smithi146=x' 2024-09-11T07:57:13.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:13 smithi102 ceph-mon[29849]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T07:57:13.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:13 smithi102 ceph-mon[29849]: Reconfiguring daemon mon.a on smithi102 2024-09-11T07:57:13.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:13 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:13.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2263666564' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:13.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:13 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:13.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:13 smithi102 ceph-mon[41371]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T07:57:13.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:13 smithi102 ceph-mon[41371]: Reconfiguring daemon mon.a on smithi102 2024-09-11T07:57:13.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:13 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:13.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2263666564' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:13.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:13 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:13.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:13 smithi146 ceph-mon[33158]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T07:57:13.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:13 smithi146 ceph-mon[33158]: Reconfiguring daemon mon.a on smithi102 2024-09-11T07:57:13.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:13 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:57:13.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2263666564' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:13.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:13 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:14.014 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:57:14.744 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:14 smithi146 ceph-mon[33158]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:14.744 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:14 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:14.744 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:14 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:14.745 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:14 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:57:14.745 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:14 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:57:14.745 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:14 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:14.829 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:14 smithi102 ceph-mon[41371]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:14.829 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:14 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:14.829 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:14 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:14.829 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:14 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:57:14.829 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:14 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:57:14.829 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:14 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:14.830 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:14 smithi102 ceph-mon[29849]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:14.830 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:14 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:14.830 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:14 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:14.830 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:14 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:57:14.830 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:14 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:57:14.830 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:14 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:15.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:15 smithi102 ceph-mon[29849]: Reconfiguring mon.c (monmap changed)... 2024-09-11T07:57:15.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:15 smithi102 ceph-mon[29849]: Reconfiguring daemon mon.c on smithi102 2024-09-11T07:57:15.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:15 smithi102 ceph-mon[41371]: Reconfiguring mon.c (monmap changed)... 2024-09-11T07:57:15.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:15 smithi102 ceph-mon[41371]: Reconfiguring daemon mon.c on smithi102 2024-09-11T07:57:15.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.637+0000 7f546fc3b640 1 -- 172.21.15.146:0/3536010181 >> v1:172.21.15.102:6789/0 conn(0x7f54680fc2f0 legacy=0x7f54680fc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:15.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.637+0000 7f546fc3b640 1 -- 172.21.15.146:0/3536010181 shutdown_connections 2024-09-11T07:57:15.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.637+0000 7f546fc3b640 1 -- 172.21.15.146:0/3536010181 >> 172.21.15.146:0/3536010181 conn(0x7f54680f7a20 msgr2=0x7f54680f9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:57:15.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.637+0000 7f546fc3b640 1 -- 172.21.15.146:0/3536010181 shutdown_connections 2024-09-11T07:57:15.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.637+0000 7f546fc3b640 1 -- 172.21.15.146:0/3536010181 wait complete. 2024-09-11T07:57:15.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.638+0000 7f546fc3b640 1 Processor -- start 2024-09-11T07:57:15.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.638+0000 7f546fc3b640 1 -- start start 2024-09-11T07:57:15.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.638+0000 7f546fc3b640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5468119560 con 0x7f546810e2d0 2024-09-11T07:57:15.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.638+0000 7f546fc3b640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f546811a760 con 0x7f54680fc2f0 2024-09-11T07:57:15.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.638+0000 7f546fc3b640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f546811b960 con 0x7f5468115860 2024-09-11T07:57:15.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.638+0000 7f546d9d7640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f54680fc2f0 0x7f546810dbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:36878/0 (socket says 172.21.15.146:36878) 2024-09-11T07:57:15.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.638+0000 7f546d9d7640 1 -- 172.21.15.146:0/3935493130 learned_addr learned my addr 172.21.15.146:0/3935493130 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-11T07:57:15.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.639+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1492697300 0 0) 0x7f5468119560 con 0x7f546810e2d0 2024-09-11T07:57:15.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.639+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5454003b80 con 0x7f546810e2d0 2024-09-11T07:57:15.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.639+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1960575084 0 0) 0x7f546811a760 con 0x7f54680fc2f0 2024-09-11T07:57:15.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.639+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5468119560 con 0x7f54680fc2f0 2024-09-11T07:57:15.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.639+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162114799 0 0) 0x7f546811b960 con 0x7f5468115860 2024-09-11T07:57:15.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.639+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f546811a760 con 0x7f5468115860 2024-09-11T07:57:15.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.639+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3947618622 0 0) 0x7f5454003b80 con 0x7f546810e2d0 2024-09-11T07:57:15.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.639+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f546811b960 con 0x7f546810e2d0 2024-09-11T07:57:15.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.639+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f5464001840 con 0x7f546810e2d0 2024-09-11T07:57:15.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.639+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3661336551 0 0) 0x7f5468119560 con 0x7f54680fc2f0 2024-09-11T07:57:15.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.639+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5454003b80 con 0x7f54680fc2f0 2024-09-11T07:57:15.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.639+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1615936755 0 0) 0x7f546811a760 con 0x7f5468115860 2024-09-11T07:57:15.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.639+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5468119560 con 0x7f5468115860 2024-09-11T07:57:15.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.639+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f545c0020c0 con 0x7f54680fc2f0 2024-09-11T07:57:15.669 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.639+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f5460001a40 con 0x7f5468115860 2024-09-11T07:57:15.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.639+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 391342721 0 0) 0x7f546811b960 con 0x7f546810e2d0 2024-09-11T07:57:15.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.639+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 >> v1:172.21.15.102:6790/0 conn(0x7f5468115860 legacy=0x7f5468117c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:15.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.639+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 >> v1:172.21.15.146:6789/0 conn(0x7f54680fc2f0 legacy=0x7f546810dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:15.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.639+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546811cb60 con 0x7f546810e2d0 2024-09-11T07:57:15.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.639+0000 7f546fc3b640 1 -- 172.21.15.146:0/3935493130 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5468119750 con 0x7f546810e2d0 2024-09-11T07:57:15.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.639+0000 7f546fc3b640 1 -- 172.21.15.146:0/3935493130 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5468119c20 con 0x7f546810e2d0 2024-09-11T07:57:15.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.639+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f54640023d0 con 0x7f546810e2d0 2024-09-11T07:57:15.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.639+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f5464003230 con 0x7f546810e2d0 2024-09-11T07:57:15.670 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.640+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 1600948874 0 0) 0x7f546400e5f0 con 0x7f546810e2d0 2024-09-11T07:57:15.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.640+0000 7f546fc3b640 1 -- 172.21.15.146:0/3935493130 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f544c005040 con 0x7f546810e2d0 2024-09-11T07:57:15.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.640+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (unknown 1709346042 0 0) 0x7f5464044e60 con 0x7f546810e2d0 2024-09-11T07:57:15.671 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.645+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f546400fe90 con 0x7f546810e2d0 2024-09-11T07:57:15.671 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:15 smithi146 ceph-mon[33158]: Reconfiguring mon.c (monmap changed)... 2024-09-11T07:57:15.671 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:15 smithi146 ceph-mon[33158]: Reconfiguring daemon mon.c on smithi102 2024-09-11T07:57:15.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.817+0000 7f546fc3b640 1 -- 172.21.15.146:0/3935493130 --> v1:172.21.15.102:6800/2950771867 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi102=y;smithi146=x", "target": ["mon-mgr", ""]}) v1 -- 0x7f544c000bc0 con 0x7f545403aa40 2024-09-11T07:57:15.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.821+0000 7f545a7fc640 1 -- 172.21.15.146:0/3935493130 <== mgr.14150 v1:172.21.15.102:6800/2950771867 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 325935098) 0x7f544c000bc0 con 0x7f545403aa40 2024-09-11T07:57:15.822 INFO:teuthology.orchestra.run.smithi146.stdout:Scheduled mgr update... 2024-09-11T07:57:15.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.823+0000 7f546fc3b640 1 -- 172.21.15.146:0/3935493130 >> v1:172.21.15.102:6800/2950771867 conn(0x7f545403aa40 legacy=0x7f545403cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:15.824 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.823+0000 7f546fc3b640 1 -- 172.21.15.146:0/3935493130 >> v1:172.21.15.102:6789/0 conn(0x7f546810e2d0 legacy=0x7f5468114140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:15.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.823+0000 7f546fc3b640 1 -- 172.21.15.146:0/3935493130 shutdown_connections 2024-09-11T07:57:15.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.823+0000 7f546fc3b640 1 -- 172.21.15.146:0/3935493130 >> 172.21.15.146:0/3935493130 conn(0x7f54680f7a20 msgr2=0x7f54680fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:57:15.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.823+0000 7f546fc3b640 1 -- 172.21.15.146:0/3935493130 shutdown_connections 2024-09-11T07:57:15.825 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:57:15.824+0000 7f546fc3b640 1 -- 172.21.15.146:0/3935493130 wait complete. 2024-09-11T07:57:16.418 DEBUG:teuthology.orchestra.run.smithi146:mgr.x> sudo journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@mgr.x.service 2024-09-11T07:57:16.421 INFO:tasks.cephadm:Deploying OSDs... 2024-09-11T07:57:16.421 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-11T07:57:16.421 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T07:57:16.440 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T07:57:16.440 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_1 2024-09-11T07:57:16.497 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T07:57:16.498 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:57:16.498 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 981 Links: 1 2024-09-11T07:57:16.498 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:57:16.498 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:57:16.498 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-11 07:57:04.166935803 +0000 2024-09-11T07:57:16.498 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-11 07:52:43.627442740 +0000 2024-09-11T07:57:16.498 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-11 07:52:43.627442740 +0000 2024-09-11T07:57:16.498 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-11 07:52:43.627442740 +0000 2024-09-11T07:57:16.498 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T07:57:16.566 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-11T07:57:16.566 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-11T07:57:16.566 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000266865 s, 1.9 MB/s 2024-09-11T07:57:16.567 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T07:57:16.626 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_2 2024-09-11T07:57:16.682 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T07:57:16.682 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:57:16.682 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 979 Links: 1 2024-09-11T07:57:16.682 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:57:16.682 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:57:16.683 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-11 07:57:04.166935803 +0000 2024-09-11T07:57:16.683 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-11 07:52:43.627442740 +0000 2024-09-11T07:57:16.683 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-11 07:52:43.627442740 +0000 2024-09-11T07:57:16.683 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-11 07:52:43.627442740 +0000 2024-09-11T07:57:16.683 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T07:57:16.748 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-11T07:57:16.748 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-11T07:57:16.748 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000236647 s, 2.2 MB/s 2024-09-11T07:57:16.749 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T07:57:16.807 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_3 2024-09-11T07:57:16.864 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T07:57:16.864 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:57:16.864 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 967 Links: 1 2024-09-11T07:57:16.864 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:57:16.864 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:57:16.865 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-11 07:57:04.167935782 +0000 2024-09-11T07:57:16.865 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-11 07:52:43.626442876 +0000 2024-09-11T07:57:16.865 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-11 07:52:43.626442876 +0000 2024-09-11T07:57:16.865 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-11 07:52:43.626442876 +0000 2024-09-11T07:57:16.865 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T07:57:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:16 smithi102 ceph-mon[41371]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:16 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:16 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:16 smithi102 ceph-mon[41371]: Reconfiguring mon.b (monmap changed)... 2024-09-11T07:57:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:16 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:57:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:16 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:57:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:16 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:16 smithi102 ceph-mon[41371]: Reconfiguring daemon mon.b on smithi146 2024-09-11T07:57:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:16 smithi102 ceph-mon[41371]: from='client.14205 v1:172.21.15.146:0/3935493130' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi102=y;smithi146=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:16 smithi102 ceph-mon[41371]: Saving service mgr spec with placement smithi102=y;smithi146=x;count:2 2024-09-11T07:57:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:16 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:16.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:16 smithi102 ceph-mon[29849]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:16.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:16 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:16.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:16 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:16.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:16 smithi102 ceph-mon[29849]: Reconfiguring mon.b (monmap changed)... 2024-09-11T07:57:16.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:16 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:57:16.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:16 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:57:16.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:16 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:16.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:16 smithi102 ceph-mon[29849]: Reconfiguring daemon mon.b on smithi146 2024-09-11T07:57:16.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:16 smithi102 ceph-mon[29849]: from='client.14205 v1:172.21.15.146:0/3935493130' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi102=y;smithi146=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:16.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:16 smithi102 ceph-mon[29849]: Saving service mgr spec with placement smithi102=y;smithi146=x;count:2 2024-09-11T07:57:16.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:16 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:16.921 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-11T07:57:16.921 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-11T07:57:16.921 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000202732 s, 2.5 MB/s 2024-09-11T07:57:16.922 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T07:57:16.978 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_4 2024-09-11T07:57:17.034 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T07:57:17.034 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:57:17.034 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 980 Links: 1 2024-09-11T07:57:17.034 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:57:17.034 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:57:17.034 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-11 07:57:04.167935782 +0000 2024-09-11T07:57:17.035 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-11 07:52:43.627442740 +0000 2024-09-11T07:57:17.035 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-11 07:52:43.627442740 +0000 2024-09-11T07:57:17.035 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-11 07:52:43.627442740 +0000 2024-09-11T07:57:17.035 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T07:57:17.098 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-11T07:57:17.098 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-11T07:57:17.098 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.00032741 s, 1.6 MB/s 2024-09-11T07:57:17.099 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T07:57:17.155 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-11T07:57:17.155 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T07:57:17.172 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T07:57:17.172 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_1 2024-09-11T07:57:17.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:16 smithi146 ceph-mon[33158]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:17.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:16 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:17.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:16 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:17.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:16 smithi146 ceph-mon[33158]: Reconfiguring mon.b (monmap changed)... 2024-09-11T07:57:17.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:16 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:57:17.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:16 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:57:17.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:16 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:17.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:16 smithi146 ceph-mon[33158]: Reconfiguring daemon mon.b on smithi146 2024-09-11T07:57:17.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:16 smithi146 ceph-mon[33158]: from='client.14205 v1:172.21.15.146:0/3935493130' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi102=y;smithi146=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:17.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:16 smithi146 ceph-mon[33158]: Saving service mgr spec with placement smithi102=y;smithi146=x;count:2 2024-09-11T07:57:17.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:16 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:17.200 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T07:57:17.200 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:57:17.200 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 964 Links: 1 2024-09-11T07:57:17.200 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:57:17.200 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:57:17.201 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-09-11 07:56:55.184471402 +0000 2024-09-11T07:57:17.201 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-09-11 07:52:43.517224309 +0000 2024-09-11T07:57:17.201 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-09-11 07:52:43.517224309 +0000 2024-09-11T07:57:17.201 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-09-11 07:52:43.517224309 +0000 2024-09-11T07:57:17.201 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T07:57:17.265 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-09-11T07:57:17.265 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-09-11T07:57:17.265 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000277132 s, 1.8 MB/s 2024-09-11T07:57:17.266 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T07:57:17.322 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_2 2024-09-11T07:57:17.377 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T07:57:17.377 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:57:17.377 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 982 Links: 1 2024-09-11T07:57:17.378 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:57:17.378 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:57:17.378 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-09-11 07:56:55.185471387 +0000 2024-09-11T07:57:17.378 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-09-11 07:52:43.518224214 +0000 2024-09-11T07:57:17.378 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-09-11 07:52:43.518224214 +0000 2024-09-11T07:57:17.378 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-09-11 07:52:43.518224214 +0000 2024-09-11T07:57:17.378 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T07:57:17.440 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-09-11T07:57:17.441 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-09-11T07:57:17.441 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000198016 s, 2.6 MB/s 2024-09-11T07:57:17.442 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T07:57:17.500 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_3 2024-09-11T07:57:17.555 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T07:57:17.555 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:57:17.555 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 978 Links: 1 2024-09-11T07:57:17.556 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:57:17.556 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:57:17.556 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-09-11 07:56:55.185471387 +0000 2024-09-11T07:57:17.556 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-09-11 07:52:43.517224309 +0000 2024-09-11T07:57:17.556 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-09-11 07:52:43.517224309 +0000 2024-09-11T07:57:17.556 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-09-11 07:52:43.517224309 +0000 2024-09-11T07:57:17.556 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T07:57:17.619 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-09-11T07:57:17.619 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-09-11T07:57:17.619 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000221738 s, 2.3 MB/s 2024-09-11T07:57:17.620 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T07:57:17.676 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_4 2024-09-11T07:57:17.732 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T07:57:17.733 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:57:17.733 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 974 Links: 1 2024-09-11T07:57:17.733 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:57:17.733 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:57:17.733 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-09-11 07:56:55.185471387 +0000 2024-09-11T07:57:17.733 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-09-11 07:52:43.517224309 +0000 2024-09-11T07:57:17.733 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-09-11 07:52:43.517224309 +0000 2024-09-11T07:57:17.733 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-09-11 07:52:43.517224309 +0000 2024-09-11T07:57:17.733 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T07:57:17.797 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-09-11T07:57:17.797 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-09-11T07:57:17.797 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000178546 s, 2.9 MB/s 2024-09-11T07:57:17.798 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T07:57:17.854 INFO:tasks.cephadm:Deploying osd.0 on smithi102 with /dev/vg_nvme/lv_4... 2024-09-11T07:57:17.855 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-11T07:57:18.088 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:57:18.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:18 smithi146 ceph-mon[33158]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:18.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:18.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:18.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:18.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:18.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:18.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:18.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:57:18.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T07:57:18.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:57:18.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:18.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:18 smithi146 ceph-mon[33158]: Deploying daemon mgr.x on smithi146 2024-09-11T07:57:18.877 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[29849]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:18.878 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:18.878 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:18.878 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:18.878 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:18.878 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:18.878 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:18.878 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:57:18.878 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T07:57:18.878 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:57:18.878 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:18.878 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[29849]: Deploying daemon mgr.x on smithi146 2024-09-11T07:57:18.879 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[41371]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:18.879 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:18.879 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:18.879 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:18.879 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:18.879 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:18.879 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:18.879 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:57:18.879 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T07:57:18.879 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:57:18.880 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:18.880 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:18 smithi102 ceph-mon[41371]: Deploying daemon mgr.x on smithi146 2024-09-11T07:57:19.434 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:19 smithi146 systemd[1]: Starting Ceph mgr.x for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T07:57:19.733 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-11T07:57:19.823 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-11T07:57:19.838 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2024-09-11T07:57:19.838 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2024-09-11T07:57:19.838 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132591 s, 791 MB/s 2024-09-11T07:57:19.838 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2024-09-11T07:57:19.902 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:19 smithi146 podman[35200]: 2024-09-11 07:57:19.455805482 +0000 UTC m=+0.015530500 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:57:19.902 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:19 smithi146 podman[35200]: 2024-09-11 07:57:19.566275615 +0000 UTC m=+0.126000695 container create 8b817570c442a0552c9a5f33ca183c5ac3bbc4304d770165ae4d490afc51b8c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T07:57:19.902 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:19 smithi146 podman[35200]: 2024-09-11 07:57:19.793497829 +0000 UTC m=+0.353222848 container init 8b817570c442a0552c9a5f33ca183c5ac3bbc4304d770165ae4d490afc51b8c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:57:19.902 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:19 smithi146 podman[35200]: 2024-09-11 07:57:19.798451202 +0000 UTC m=+0.358176223 container start 8b817570c442a0552c9a5f33ca183c5ac3bbc4304d770165ae4d490afc51b8c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-11T07:57:19.902 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:19 smithi146 bash[35200]: 8b817570c442a0552c9a5f33ca183c5ac3bbc4304d770165ae4d490afc51b8c9 2024-09-11T07:57:20.184 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:19 smithi146 systemd[1]: Started Ceph mgr.x for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T07:57:20.184 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:19 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:19.910+0000 7fc7f50dbec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:57:20.184 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:19 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:19.984+0000 7fc7f50dbec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:57:20.184 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:20 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:20.053+0000 7fc7f50dbec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:57:20.426 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_4 2024-09-11T07:57:20.613 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:20 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:20.185+0000 7fc7f50dbec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:57:20.613 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:20 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:20.332+0000 7fc7f50dbec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:57:20.613 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:20 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:20.612+0000 7fc7f50dbec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:57:20.662 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:57:20.930 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:20 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:20.676+0000 7fc7f50dbec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:57:20.930 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:20 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:20.843+0000 7fc7f50dbec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:57:21.114 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:20 smithi102 ceph-mon[29849]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:21.114 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:20 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:21.114 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:20 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:21.114 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:20 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:21.114 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:20 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:21.114 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:20 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:21.115 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:20 smithi102 ceph-mon[41371]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:21.115 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:20 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:21.115 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:20 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:21.115 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:20 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:21.115 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:20 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:21.115 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:20 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:21.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:20 smithi146 ceph-mon[33158]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:21.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:20 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:21.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:20 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:21.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:20 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:21.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:20 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:21.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:20 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:22.163 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:21 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:21.701+0000 7fc7f50dbec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:57:22.163 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:21 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:21.764+0000 7fc7f50dbec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:57:22.382 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:22 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:22.382 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:22 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:22.382 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:22 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:22.382 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:22 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:22.382 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:22 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:22.382 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:22 smithi102 ceph-mon[29849]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T07:57:22.382 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:22 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:57:22.382 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:22 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:57:22.383 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:22 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:22.383 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:22 smithi102 ceph-mon[29849]: Reconfiguring daemon mgr.y on smithi102 2024-09-11T07:57:22.383 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:22 smithi102 ceph-mon[29849]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:22.383 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:22 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:22.383 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:22 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:22.383 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:22 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:22.383 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:22 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:22.383 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:22 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:22.383 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:22 smithi102 ceph-mon[41371]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T07:57:22.383 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:22 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:57:22.384 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:22 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:57:22.384 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:22 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:22.384 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:22 smithi102 ceph-mon[41371]: Reconfiguring daemon mgr.y on smithi102 2024-09-11T07:57:22.384 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:22 smithi102 ceph-mon[41371]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:22 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:22 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:22 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:22 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:22 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:22 smithi146 ceph-mon[33158]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T07:57:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:22 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:57:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:22 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:57:22.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:22 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:22.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:22 smithi146 ceph-mon[33158]: Reconfiguring daemon mgr.y on smithi102 2024-09-11T07:57:22.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:22 smithi146 ceph-mon[33158]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:22.756 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:22 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:22.471+0000 7fc7f50dbec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:57:22.756 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:22 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:22.580+0000 7fc7f50dbec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:57:22.756 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:22 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:22.697+0000 7fc7f50dbec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:57:22.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.913+0000 7f88aee38640 1 -- 172.21.15.102:0/3757883942 >> v1:172.21.15.102:6789/0 conn(0x7f88a80ff960 legacy=0x7f88a8101d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:22.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.913+0000 7f88aee38640 1 -- 172.21.15.102:0/3757883942 shutdown_connections 2024-09-11T07:57:22.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.913+0000 7f88aee38640 1 -- 172.21.15.102:0/3757883942 >> 172.21.15.102:0/3757883942 conn(0x7f88a80f6c20 msgr2=0x7f88a80f9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:57:22.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.913+0000 7f88aee38640 1 -- 172.21.15.102:0/3757883942 shutdown_connections 2024-09-11T07:57:22.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.913+0000 7f88aee38640 1 -- 172.21.15.102:0/3757883942 wait complete. 2024-09-11T07:57:22.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.914+0000 7f88aee38640 1 Processor -- start 2024-09-11T07:57:22.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.914+0000 7f88aee38640 1 -- start start 2024-09-11T07:57:22.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.914+0000 7f88aee38640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f88a810d050 con 0x7f88a8103580 2024-09-11T07:57:22.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.914+0000 7f88aee38640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f88a8118850 con 0x7f88a80ff960 2024-09-11T07:57:22.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.914+0000 7f88aee38640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f88a81199f0 con 0x7f88a80fb460 2024-09-11T07:57:22.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.914+0000 7f88acbd4640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7f88a80fb460 0x7f88a810c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:54704/0 (socket says 172.21.15.102:54704) 2024-09-11T07:57:22.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.914+0000 7f88ad3d5640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f88a8103580 0x7f88a8116120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:53022/0 (socket says 172.21.15.102:53022) 2024-09-11T07:57:22.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.914+0000 7f88acbd4640 1 -- 172.21.15.102:0/3172293557 learned_addr learned my addr 172.21.15.102:0/3172293557 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:57:22.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.915+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3759095706 0 0) 0x7f88a810d050 con 0x7f88a8103580 2024-09-11T07:57:22.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.915+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88980034e0 con 0x7f88a8103580 2024-09-11T07:57:22.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.915+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4215104589 0 0) 0x7f88a81199f0 con 0x7f88a80fb460 2024-09-11T07:57:22.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.915+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88a810d050 con 0x7f88a80fb460 2024-09-11T07:57:22.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.915+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2633395039 0 0) 0x7f88a8118850 con 0x7f88a80ff960 2024-09-11T07:57:22.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.915+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88a81199f0 con 0x7f88a80ff960 2024-09-11T07:57:22.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.915+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4122192380 0 0) 0x7f88980034e0 con 0x7f88a8103580 2024-09-11T07:57:22.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.915+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f88a8118850 con 0x7f88a8103580 2024-09-11T07:57:22.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.915+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f88a0001c40 con 0x7f88a8103580 2024-09-11T07:57:22.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.915+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3433809958 0 0) 0x7f88a810d050 con 0x7f88a80fb460 2024-09-11T07:57:22.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.915+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f88980034e0 con 0x7f88a80fb460 2024-09-11T07:57:22.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.915+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2906306819 0 0) 0x7f88a81199f0 con 0x7f88a80ff960 2024-09-11T07:57:22.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.915+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f88a810d050 con 0x7f88a80ff960 2024-09-11T07:57:22.919 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.915+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f889c0018b0 con 0x7f88a80fb460 2024-09-11T07:57:22.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.915+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f8894001ec0 con 0x7f88a80ff960 2024-09-11T07:57:22.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.915+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3942193506 0 0) 0x7f88a8118850 con 0x7f88a8103580 2024-09-11T07:57:22.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.915+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 >> v1:172.21.15.102:6790/0 conn(0x7f88a80fb460 legacy=0x7f88a810c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:22.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.915+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 >> v1:172.21.15.146:6789/0 conn(0x7f88a80ff960 legacy=0x7f88a81129f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:22.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.915+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a811abd0 con 0x7f88a8103580 2024-09-11T07:57:22.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.915+0000 7f88aee38640 1 -- 172.21.15.102:0/3172293557 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a81178a0 con 0x7f88a8103580 2024-09-11T07:57:22.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.915+0000 7f88aee38640 1 -- 172.21.15.102:0/3172293557 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a8117d70 con 0x7f88a8103580 2024-09-11T07:57:22.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.915+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f88a0002330 con 0x7f88a8103580 2024-09-11T07:57:22.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.915+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f88a0005e00 con 0x7f88a8103580 2024-09-11T07:57:22.920 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.916+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 1600948874 0 0) 0x7f88a0011140 con 0x7f88a8103580 2024-09-11T07:57:22.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.916+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (unknown 1709346042 0 0) 0x7f88a0047a90 con 0x7f88a8103580 2024-09-11T07:57:22.921 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.916+0000 7f88aee38640 1 -- 172.21.15.102:0/3172293557 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f888c005040 con 0x7f88a8103580 2024-09-11T07:57:22.922 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:22.921+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f88a00129e0 con 0x7f88a8103580 2024-09-11T07:57:23.088 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:23.087+0000 7f88aee38640 1 -- 172.21.15.102:0/3172293557 --> v1:172.21.15.102:6800/2950771867 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f888c000bc0 con 0x7f889803a840 2024-09-11T07:57:23.135 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:22 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:22.755+0000 7fc7f50dbec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:57:23.135 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:22 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:22.880+0000 7fc7f50dbec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:57:23.135 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:22 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:22.946+0000 7fc7f50dbec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:57:23.135 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:23 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:23.002+0000 7fc7f50dbec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:57:23.135 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:23 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:23.134+0000 7fc7f50dbec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:57:23.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:23 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:57:23.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:23 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:57:23.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:23 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:23.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:23 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:57:23.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:23 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:57:23.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:23 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:23.402 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:23 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:57:23.402 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:23 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:57:23.402 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:23 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:23.402 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:23 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:23.197+0000 7fc7f50dbec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:57:23.402 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:23 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:23.261+0000 7fc7f50dbec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:57:23.403 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:23 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:23.341+0000 7fc7f50dbec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:57:23.403 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:23 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:23.401+0000 7fc7f50dbec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:57:23.684 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:23 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:23.458+0000 7fc7f50dbec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:57:24.447 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:24.445+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mgrmap(e 14) v1 ==== 90690+0+0 (unknown 2220093901 0 0) 0x7f88a0012660 con 0x7f88a8103580 2024-09-11T07:57:24.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[41371]: from='client.14217 v1:172.21.15.102:0/3172293557' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:24.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:24.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:24.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:24.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:24.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:24.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:24.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[41371]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:24.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:24.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[41371]: Standby manager daemon x started 2024-09-11T07:57:24.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[41371]: from='mgr.? v1:172.21.15.146:0/3773226598' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:57:24.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[41371]: from='mgr.? v1:172.21.15.146:0/3773226598' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:57:24.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[41371]: from='mgr.? v1:172.21.15.146:0/3773226598' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:57:24.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[41371]: from='mgr.? v1:172.21.15.146:0/3773226598' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:57:24.494 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[29849]: from='client.14217 v1:172.21.15.102:0/3172293557' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:24.494 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:24.494 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:24.494 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:24.494 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:24.494 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:24.494 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:24.495 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[29849]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:24.495 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:24.495 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[29849]: Standby manager daemon x started 2024-09-11T07:57:24.495 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[29849]: from='mgr.? v1:172.21.15.146:0/3773226598' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:57:24.495 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[29849]: from='mgr.? v1:172.21.15.146:0/3773226598' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:57:24.495 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[29849]: from='mgr.? v1:172.21.15.146:0/3773226598' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:57:24.495 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:24 smithi102 ceph-mon[29849]: from='mgr.? v1:172.21.15.146:0/3773226598' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:57:24.684 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 07:57:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T07:57:24.191+0000 7fc7f50dbec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:57:24.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:24 smithi146 ceph-mon[33158]: from='client.14217 v1:172.21.15.102:0/3172293557' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:24.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:24 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:24.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:24 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:24.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:24 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:24.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:24 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:24.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:24 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:24.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:24 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:24.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:24 smithi146 ceph-mon[33158]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:24.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:24 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:24.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:24 smithi146 ceph-mon[33158]: Standby manager daemon x started 2024-09-11T07:57:24.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:24 smithi146 ceph-mon[33158]: from='mgr.? v1:172.21.15.146:0/3773226598' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:57:24.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:24 smithi146 ceph-mon[33158]: from='mgr.? v1:172.21.15.146:0/3773226598' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:57:24.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:24 smithi146 ceph-mon[33158]: from='mgr.? v1:172.21.15.146:0/3773226598' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:57:24.686 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:24 smithi146 ceph-mon[33158]: from='mgr.? v1:172.21.15.146:0/3773226598' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:57:25.743 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:25 smithi102 ceph-mon[29849]: mgrmap e14: y(active, since 61s), standbys: x 2024-09-11T07:57:25.743 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:25 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:57:25.743 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1184293796' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08248393-1eca-4c59-b577-e0ee4ed4d6e3"}]: dispatch 2024-09-11T07:57:25.744 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1184293796' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "08248393-1eca-4c59-b577-e0ee4ed4d6e3"}]': finished 2024-09-11T07:57:25.744 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:25 smithi102 ceph-mon[29849]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T07:57:25.744 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:25 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:57:25.744 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:25 smithi102 ceph-mon[41371]: mgrmap e14: y(active, since 61s), standbys: x 2024-09-11T07:57:25.744 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:25 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:57:25.744 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1184293796' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08248393-1eca-4c59-b577-e0ee4ed4d6e3"}]: dispatch 2024-09-11T07:57:25.744 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1184293796' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "08248393-1eca-4c59-b577-e0ee4ed4d6e3"}]': finished 2024-09-11T07:57:25.744 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:25 smithi102 ceph-mon[41371]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T07:57:25.744 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:25 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:57:25.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:25 smithi146 ceph-mon[33158]: mgrmap e14: y(active, since 61s), standbys: x 2024-09-11T07:57:25.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:25 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:57:25.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1184293796' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08248393-1eca-4c59-b577-e0ee4ed4d6e3"}]: dispatch 2024-09-11T07:57:25.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1184293796' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "08248393-1eca-4c59-b577-e0ee4ed4d6e3"}]': finished 2024-09-11T07:57:25.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:25 smithi146 ceph-mon[33158]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T07:57:25.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:25 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:57:26.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:26 smithi102 ceph-mon[29849]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:26.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2719467971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:57:26.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:26 smithi102 ceph-mon[41371]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:26.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2719467971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:57:26.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:26 smithi146 ceph-mon[33158]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:26.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2719467971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:57:28.768 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:28 smithi102 ceph-mon[41371]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:28.769 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:28 smithi102 ceph-mon[29849]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:28.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:28 smithi146 ceph-mon[33158]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:30.519 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:30 smithi102 ceph-mon[29849]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:30.520 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:30 smithi102 ceph-mon[41371]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:30 smithi146 ceph-mon[33158]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:31.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:31 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T07:57:31.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:31 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:31.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:31 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T07:57:31.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:31 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:31 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T07:57:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:31 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:32.604 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:32 smithi102 ceph-mon[29849]: Deploying daemon osd.0 on smithi102 2024-09-11T07:57:32.604 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:32 smithi102 ceph-mon[29849]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:32.604 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:32 smithi102 ceph-mon[41371]: Deploying daemon osd.0 on smithi102 2024-09-11T07:57:32.604 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:32 smithi102 ceph-mon[41371]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:32.835 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:32 smithi146 ceph-mon[33158]: Deploying daemon osd.0 on smithi102 2024-09-11T07:57:32.835 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:32 smithi146 ceph-mon[33158]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:34.842 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:34 smithi102 ceph-mon[29849]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:34.842 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:34 smithi102 ceph-mon[41371]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:34.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:34 smithi146 ceph-mon[33158]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:36.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:36 smithi102 ceph-mon[29849]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:36.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:36 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:36.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:36 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:36.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:36 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:36.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:36 smithi102 ceph-mon[41371]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:36.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:36 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:36.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:36 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:36.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:36 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:36.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:36 smithi146 ceph-mon[33158]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:36.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:36 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:36.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:36 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:36.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:36 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:38.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:38 smithi102 ceph-mon[29849]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:38.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:38 smithi102 ceph-mon[41371]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:38.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:38 smithi146 ceph-mon[33158]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:39.213 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:39.211+0000 7f88a57fa640 1 -- 172.21.15.102:0/3172293557 <== mgr.14150 v1:172.21.15.102:6800/2950771867 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 1810881232) 0x7f889c009030 con 0x7f889803a840 2024-09-11T07:57:39.213 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 0 on host 'smithi102' 2024-09-11T07:57:39.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:39.214+0000 7f88aee38640 1 -- 172.21.15.102:0/3172293557 >> v1:172.21.15.102:6800/2950771867 conn(0x7f889803a840 legacy=0x7f889803ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:39.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:39.214+0000 7f88aee38640 1 -- 172.21.15.102:0/3172293557 >> v1:172.21.15.102:6789/0 conn(0x7f88a8103580 legacy=0x7f88a8116120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:39.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:39.214+0000 7f88aee38640 1 -- 172.21.15.102:0/3172293557 shutdown_connections 2024-09-11T07:57:39.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:39.214+0000 7f88aee38640 1 -- 172.21.15.102:0/3172293557 >> 172.21.15.102:0/3172293557 conn(0x7f88a80f6c20 msgr2=0x7f88a8066410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:57:39.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:39.214+0000 7f88aee38640 1 -- 172.21.15.102:0/3172293557 shutdown_connections 2024-09-11T07:57:39.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:39.214+0000 7f88aee38640 1 -- 172.21.15.102:0/3172293557 wait complete. 2024-09-11T07:57:39.775 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:39 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:39.776 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:39 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:39.776 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:39 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:39.776 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:39 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:39.776 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:39 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:39.776 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:39 smithi102 ceph-mon[41371]: from='osd.0 v1:172.21.15.102:6801/444496098' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:57:39.776 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:39 smithi102 ceph-mon[41371]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:57:39.776 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:39 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:39.776 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:39 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:39.776 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:39 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:39.776 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:39 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:39.777 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:39 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:39.777 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:39 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:39.777 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:39 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:39.777 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:39 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:39.777 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:39 smithi102 ceph-mon[29849]: from='osd.0 v1:172.21.15.102:6801/444496098' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:57:39.777 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:39 smithi102 ceph-mon[29849]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:57:39.777 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:39 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:39.777 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:39 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:39.777 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:39 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:39.802 DEBUG:teuthology.orchestra.run.smithi102:osd.0> sudo journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.0.service 2024-09-11T07:57:39.804 INFO:tasks.cephadm:Deploying osd.1 on smithi102 with /dev/vg_nvme/lv_3... 2024-09-11T07:57:39.804 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-11T07:57:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:39 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:39 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:39 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:39 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:39 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:39 smithi146 ceph-mon[33158]: from='osd.0 v1:172.21.15.102:6801/444496098' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:57:39.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:39 smithi146 ceph-mon[33158]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:57:39.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:39 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:57:39.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:39 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:39.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:39 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:41.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:40 smithi146 ceph-mon[33158]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:41.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:40 smithi146 ceph-mon[33158]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T07:57:41.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:40 smithi146 ceph-mon[33158]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T07:57:41.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:40 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:57:41.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:40 smithi146 ceph-mon[33158]: from='osd.0 v1:172.21.15.102:6801/444496098' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-11T07:57:41.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:40 smithi146 ceph-mon[33158]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-11T07:57:41.186 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:40 smithi102 ceph-mon[29849]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:41.186 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:40 smithi102 ceph-mon[29849]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T07:57:41.186 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:40 smithi102 ceph-mon[29849]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T07:57:41.186 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:40 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:57:41.186 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:40 smithi102 ceph-mon[29849]: from='osd.0 v1:172.21.15.102:6801/444496098' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-11T07:57:41.186 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:40 smithi102 ceph-mon[29849]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-11T07:57:41.187 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:40 smithi102 ceph-mon[41371]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:41.187 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:40 smithi102 ceph-mon[41371]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T07:57:41.187 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:40 smithi102 ceph-mon[41371]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T07:57:41.187 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:40 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:57:41.187 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:40 smithi102 ceph-mon[41371]: from='osd.0 v1:172.21.15.102:6801/444496098' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-11T07:57:41.187 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:40 smithi102 ceph-mon[41371]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-11T07:57:41.187 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 07:57:40 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-0[45637]: 2024-09-11T07:57:40.921+0000 7f4e1ef01640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T07:57:41.187 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 07:57:40 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-0[45637]: 2024-09-11T07:57:40.923+0000 7f4e19d38640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:57:41.382 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:57:42.118 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:41 smithi102 ceph-mon[41371]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-11T07:57:42.118 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:41 smithi102 ceph-mon[41371]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T07:57:42.118 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:41 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:57:42.118 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:41 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:57:42.119 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:41 smithi102 ceph-mon[29849]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-11T07:57:42.119 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:41 smithi102 ceph-mon[29849]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T07:57:42.119 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:41 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:57:42.119 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:41 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:57:42.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:41 smithi146 ceph-mon[33158]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-11T07:57:42.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:41 smithi146 ceph-mon[33158]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T07:57:42.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:41 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:57:42.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:41 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:57:43.113 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[29849]: purged_snaps scrub starts 2024-09-11T07:57:43.113 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[29849]: purged_snaps scrub ok 2024-09-11T07:57:43.113 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[29849]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:43.113 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[29849]: osd.0 v1:172.21.15.102:6801/444496098 boot 2024-09-11T07:57:43.113 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[29849]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T07:57:43.113 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:57:43.113 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:43.113 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:43.113 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:57:43.113 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:43.114 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:43.114 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:43.114 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:43.114 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[41371]: purged_snaps scrub starts 2024-09-11T07:57:43.114 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[41371]: purged_snaps scrub ok 2024-09-11T07:57:43.114 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[41371]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:43.114 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[41371]: osd.0 v1:172.21.15.102:6801/444496098 boot 2024-09-11T07:57:43.114 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[41371]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T07:57:43.114 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:57:43.114 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:43.114 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:43.115 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:57:43.115 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:43.115 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:43.115 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:43.115 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:42 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:42 smithi146 ceph-mon[33158]: purged_snaps scrub starts 2024-09-11T07:57:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:42 smithi146 ceph-mon[33158]: purged_snaps scrub ok 2024-09-11T07:57:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:42 smithi146 ceph-mon[33158]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:57:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:42 smithi146 ceph-mon[33158]: osd.0 v1:172.21.15.102:6801/444496098 boot 2024-09-11T07:57:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:42 smithi146 ceph-mon[33158]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T07:57:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:42 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:57:43.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:42 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:43.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:42 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:43.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:42 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:57:43.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:42 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:43.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:42 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:43.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:42 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:57:43.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:42 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:57:43.764 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-11T07:57:43.843 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-11T07:57:43.857 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2024-09-11T07:57:43.857 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2024-09-11T07:57:43.857 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012342 s, 850 MB/s 2024-09-11T07:57:43.857 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2024-09-11T07:57:44.126 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:43 smithi102 ceph-mon[41371]: Detected new or changed devices on smithi102 2024-09-11T07:57:44.127 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:43 smithi102 ceph-mon[41371]: Adjusting osd_memory_target on smithi102 to 16064M 2024-09-11T07:57:44.127 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:43 smithi102 ceph-mon[41371]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T07:57:44.127 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:43 smithi102 ceph-mon[29849]: Detected new or changed devices on smithi102 2024-09-11T07:57:44.127 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:43 smithi102 ceph-mon[29849]: Adjusting osd_memory_target on smithi102 to 16064M 2024-09-11T07:57:44.127 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:43 smithi102 ceph-mon[29849]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T07:57:44.394 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_3 2024-09-11T07:57:44.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:43 smithi146 ceph-mon[33158]: Detected new or changed devices on smithi102 2024-09-11T07:57:44.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:43 smithi146 ceph-mon[33158]: Adjusting osd_memory_target on smithi102 to 16064M 2024-09-11T07:57:44.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:43 smithi146 ceph-mon[33158]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T07:57:44.633 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:57:44.996 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:44 smithi102 ceph-mon[29849]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:57:44.996 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:44 smithi102 ceph-mon[41371]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:57:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:44 smithi146 ceph-mon[33158]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:57:46.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.176+0000 7f726993b640 1 -- 172.21.15.102:0/3929642829 >> v1:172.21.15.102:6789/0 conn(0x7f72640ff960 legacy=0x7f7264101d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:46.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.176+0000 7f726993b640 1 -- 172.21.15.102:0/3929642829 shutdown_connections 2024-09-11T07:57:46.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.176+0000 7f726993b640 1 -- 172.21.15.102:0/3929642829 >> 172.21.15.102:0/3929642829 conn(0x7f72640f6c20 msgr2=0x7f72640f9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:57:46.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.176+0000 7f726993b640 1 -- 172.21.15.102:0/3929642829 shutdown_connections 2024-09-11T07:57:46.225 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.176+0000 7f726993b640 1 -- 172.21.15.102:0/3929642829 wait complete. 2024-09-11T07:57:46.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.177+0000 7f726993b640 1 Processor -- start 2024-09-11T07:57:46.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.177+0000 7f726993b640 1 -- start start 2024-09-11T07:57:46.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.177+0000 7f726993b640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7264117460 con 0x7f72640fb460 2024-09-11T07:57:46.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.177+0000 7f726993b640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7264118660 con 0x7f7264103580 2024-09-11T07:57:46.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.177+0000 7f726993b640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7264119860 con 0x7f72640ff960 2024-09-11T07:57:46.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.177+0000 7f7262ffd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f72640fb460 0x7f7264066550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:54008/0 (socket says 172.21.15.102:54008) 2024-09-11T07:57:46.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.177+0000 7f72627fc640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7f72640ff960 0x7f72641121b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:56144/0 (socket says 172.21.15.102:56144) 2024-09-11T07:57:46.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.177+0000 7f7262ffd640 1 -- 172.21.15.102:0/4022460829 learned_addr learned my addr 172.21.15.102:0/4022460829 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:57:46.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.177+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940403886 0 0) 0x7f7264119860 con 0x7f72640ff960 2024-09-11T07:57:46.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.177+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f724c0034e0 con 0x7f72640ff960 2024-09-11T07:57:46.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.177+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1315111731 0 0) 0x7f7264117460 con 0x7f72640fb460 2024-09-11T07:57:46.226 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.177+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7264119860 con 0x7f72640fb460 2024-09-11T07:57:46.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.177+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2085674678 0 0) 0x7f7264118660 con 0x7f7264103580 2024-09-11T07:57:46.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.177+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7264117460 con 0x7f7264103580 2024-09-11T07:57:46.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.178+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2145364769 0 0) 0x7f724c0034e0 con 0x7f72640ff960 2024-09-11T07:57:46.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.178+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7264118660 con 0x7f72640ff960 2024-09-11T07:57:46.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.178+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f72580015d0 con 0x7f72640ff960 2024-09-11T07:57:46.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.178+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1565070802 0 0) 0x7f7264119860 con 0x7f72640fb460 2024-09-11T07:57:46.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.178+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f724c0034e0 con 0x7f72640fb460 2024-09-11T07:57:46.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.178+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f72540018d0 con 0x7f72640fb460 2024-09-11T07:57:46.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.178+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 469863140 0 0) 0x7f7264117460 con 0x7f7264103580 2024-09-11T07:57:46.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.178+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7264119860 con 0x7f7264103580 2024-09-11T07:57:46.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.178+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f725c001bc0 con 0x7f7264103580 2024-09-11T07:57:46.227 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.178+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1743510364 0 0) 0x7f7264118660 con 0x7f72640ff960 2024-09-11T07:57:46.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.178+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 >> v1:172.21.15.146:6789/0 conn(0x7f7264103580 legacy=0x7f7264115b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:46.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.178+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 >> v1:172.21.15.102:6789/0 conn(0x7f72640fb460 legacy=0x7f7264066550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:57:46.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.178+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726411aa60 con 0x7f72640ff960 2024-09-11T07:57:46.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.178+0000 7f726993b640 1 -- 172.21.15.102:0/4022460829 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7264117650 con 0x7f72640ff960 2024-09-11T07:57:46.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.178+0000 7f726993b640 1 -- 172.21.15.102:0/4022460829 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7264117b20 con 0x7f72640ff960 2024-09-11T07:57:46.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.178+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f7258001200 con 0x7f72640ff960 2024-09-11T07:57:46.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.178+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f7258003610 con 0x7f72640ff960 2024-09-11T07:57:46.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.179+0000 7f726993b640 1 -- 172.21.15.102:0/4022460829 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7250005040 con 0x7f72640ff960 2024-09-11T07:57:46.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.179+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 14) v1 ==== 90690+0+0 (unknown 2220093901 0 0) 0x7f7258001eb0 con 0x7f72640ff960 2024-09-11T07:57:46.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.180+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(9..9 src has 1..9) v4 ==== 1537+0+0 (unknown 3268705556 0 0) 0x7f7258088cc0 con 0x7f72640ff960 2024-09-11T07:57:46.228 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.184+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f7258056d00 con 0x7f72640ff960 2024-09-11T07:57:46.351 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:57:46.350+0000 7f726993b640 1 -- 172.21.15.102:0/4022460829 --> v1:172.21.15.102:6800/2950771867 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f7250000bc0 con 0x7f724c06fbe0 2024-09-11T07:57:47.081 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:46 smithi102 ceph-mon[29849]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:57:47.081 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:46 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:57:47.081 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:46 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:57:47.081 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:46 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:47.081 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:46 smithi102 ceph-mon[41371]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:57:47.082 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:46 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:57:47.082 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:46 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:57:47.082 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:46 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:47.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:46 smithi146 ceph-mon[33158]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:57:47.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:46 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:57:47.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:46 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:57:47.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:46 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:48.205 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:47 smithi102 ceph-mon[29849]: from='client.24143 v1:172.21.15.102:0/4022460829' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:48.205 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:47 smithi102 ceph-mon[41371]: from='client.24143 v1:172.21.15.102:0/4022460829' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:48.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:47 smithi146 ceph-mon[33158]: from='client.24143 v1:172.21.15.102:0/4022460829' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:57:49.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:48 smithi102 ceph-mon[29849]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:57:49.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/459963346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "209cc4e5-6ff0-49f4-8ce4-0540ea1b5c16"}]: dispatch 2024-09-11T07:57:49.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/459963346' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "209cc4e5-6ff0-49f4-8ce4-0540ea1b5c16"}]': finished 2024-09-11T07:57:49.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:48 smithi102 ceph-mon[29849]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T07:57:49.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:48 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:57:49.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:48 smithi102 ceph-mon[41371]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:57:49.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/459963346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "209cc4e5-6ff0-49f4-8ce4-0540ea1b5c16"}]: dispatch 2024-09-11T07:57:49.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/459963346' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "209cc4e5-6ff0-49f4-8ce4-0540ea1b5c16"}]': finished 2024-09-11T07:57:49.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:48 smithi102 ceph-mon[41371]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T07:57:49.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:48 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:57:49.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:48 smithi146 ceph-mon[33158]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:57:49.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/459963346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "209cc4e5-6ff0-49f4-8ce4-0540ea1b5c16"}]: dispatch 2024-09-11T07:57:49.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/459963346' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "209cc4e5-6ff0-49f4-8ce4-0540ea1b5c16"}]': finished 2024-09-11T07:57:49.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:48 smithi146 ceph-mon[33158]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T07:57:49.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:48 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:57:50.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:49 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1588118659' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:57:50.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:49 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1588118659' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:57:50.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:49 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1588118659' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:57:51.374 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:50 smithi102 ceph-mon[29849]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:57:51.374 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:50 smithi102 ceph-mon[41371]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:57:51.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:50 smithi146 ceph-mon[33158]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:57:53.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:52 smithi102 ceph-mon[29849]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:57:53.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:52 smithi102 ceph-mon[41371]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:57:53.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:52 smithi146 ceph-mon[33158]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:57:55.217 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:54 smithi102 ceph-mon[29849]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:57:55.218 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:54 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T07:57:55.218 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:54 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:55.218 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:54 smithi102 ceph-mon[41371]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:57:55.218 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:54 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T07:57:55.218 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:54 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:54 smithi146 ceph-mon[33158]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:57:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:54 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T07:57:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:54 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:57:56.308 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:55 smithi102 ceph-mon[41371]: Deploying daemon osd.1 on smithi102 2024-09-11T07:57:56.309 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:55 smithi102 ceph-mon[29849]: Deploying daemon osd.1 on smithi102 2024-09-11T07:57:56.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:55 smithi146 ceph-mon[33158]: Deploying daemon osd.1 on smithi102 2024-09-11T07:57:57.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:56 smithi102 ceph-mon[41371]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:57:57.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:56 smithi102 ceph-mon[29849]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:57:57.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:56 smithi146 ceph-mon[33158]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:57:59.115 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:58 smithi102 ceph-mon[29849]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:57:59.116 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:58 smithi102 ceph-mon[41371]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:57:59.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:58 smithi146 ceph-mon[33158]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:58:00.075 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:59 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:58:00.075 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:59 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:00.075 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:57:59 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:00.075 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:59 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:58:00.075 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:59 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:00.076 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:57:59 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:00.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:59 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:58:00.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:59 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:00.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:57:59 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:01.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:00 smithi102 ceph-mon[29849]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:58:01.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:00 smithi102 ceph-mon[41371]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:58:01.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:00 smithi146 ceph-mon[33158]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:58:02.815 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:02.814+0000 7f724b7fe640 1 -- 172.21.15.102:0/4022460829 <== mgr.14150 v1:172.21.15.102:6800/2950771867 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 939144292) 0x7f7250000bc0 con 0x7f724c06fbe0 2024-09-11T07:58:02.816 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 1 on host 'smithi102' 2024-09-11T07:58:02.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:02.817+0000 7f726993b640 1 -- 172.21.15.102:0/4022460829 >> v1:172.21.15.102:6800/2950771867 conn(0x7f724c06fbe0 legacy=0x7f724c072090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:58:02.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:02.817+0000 7f726993b640 1 -- 172.21.15.102:0/4022460829 >> v1:172.21.15.102:6790/0 conn(0x7f72640ff960 legacy=0x7f72641121b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:58:02.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:02.817+0000 7f726993b640 1 -- 172.21.15.102:0/4022460829 shutdown_connections 2024-09-11T07:58:02.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:02.817+0000 7f726993b640 1 -- 172.21.15.102:0/4022460829 >> 172.21.15.102:0/4022460829 conn(0x7f72640f6c20 msgr2=0x7f7264106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:58:02.818 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:02.817+0000 7f726993b640 1 -- 172.21.15.102:0/4022460829 shutdown_connections 2024-09-11T07:58:02.819 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:02.817+0000 7f726993b640 1 -- 172.21.15.102:0/4022460829 wait complete. 2024-09-11T07:58:03.133 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:02 smithi102 ceph-mon[29849]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:58:03.133 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:02 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:03.133 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:02 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:03.134 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:02 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:03.134 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:02 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:58:03.134 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:02 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:03.134 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:02 smithi102 ceph-mon[29849]: from='osd.1 v1:172.21.15.102:6805/2501991548' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T07:58:03.134 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:02 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:58:03.134 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:02 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:03.134 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:02 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:03.134 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:02 smithi102 ceph-mon[41371]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:58:03.134 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:02 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:03.134 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:02 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:03.134 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:02 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:03.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:02 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:58:03.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:02 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:03.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:02 smithi102 ceph-mon[41371]: from='osd.1 v1:172.21.15.102:6805/2501991548' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T07:58:03.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:02 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:58:03.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:02 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:03.135 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:02 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:03.236 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:02 smithi146 ceph-mon[33158]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:58:03.237 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:02 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:03.237 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:02 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:03.237 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:02 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:03.237 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:02 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:58:03.237 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:02 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:03.237 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:02 smithi146 ceph-mon[33158]: from='osd.1 v1:172.21.15.102:6805/2501991548' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T07:58:03.237 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:02 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:58:03.237 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:02 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:03.237 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:02 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:03.416 DEBUG:teuthology.orchestra.run.smithi102:osd.1> sudo journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.1.service 2024-09-11T07:58:03.418 INFO:tasks.cephadm:Deploying osd.2 on smithi102 with /dev/vg_nvme/lv_2... 2024-09-11T07:58:03.419 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-11T07:58:04.261 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:58:04.381 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:04 smithi102 ceph-mon[29849]: from='osd.1 v1:172.21.15.102:6805/2501991548' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T07:58:04.382 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:04 smithi102 ceph-mon[29849]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T07:58:04.382 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:04 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:58:04.382 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:04 smithi102 ceph-mon[29849]: from='osd.1 v1:172.21.15.102:6805/2501991548' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-11T07:58:04.382 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:04 smithi102 ceph-mon[29849]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:58:04.382 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:04 smithi102 ceph-mon[41371]: from='osd.1 v1:172.21.15.102:6805/2501991548' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T07:58:04.382 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:04 smithi102 ceph-mon[41371]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T07:58:04.383 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:04 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:58:04.383 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:04 smithi102 ceph-mon[41371]: from='osd.1 v1:172.21.15.102:6805/2501991548' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-11T07:58:04.383 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:04 smithi102 ceph-mon[41371]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:58:04.383 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 07:58:04 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-1[51767]: 2024-09-11T07:58:04.043+0000 7fee10558640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T07:58:04.383 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 07:58:04 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-1[51767]: 2024-09-11T07:58:04.045+0000 7fee0938b640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:58:04.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:04 smithi146 ceph-mon[33158]: from='osd.1 v1:172.21.15.102:6805/2501991548' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T07:58:04.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:04 smithi146 ceph-mon[33158]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T07:58:04.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:04 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:58:04.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:04 smithi146 ceph-mon[33158]: from='osd.1 v1:172.21.15.102:6805/2501991548' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-11T07:58:04.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:04 smithi146 ceph-mon[33158]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:58:05.304 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:05 smithi102 ceph-mon[29849]: from='osd.1 v1:172.21.15.102:6805/2501991548' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-11T07:58:05.304 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:05 smithi102 ceph-mon[29849]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T07:58:05.305 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:05 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:58:05.305 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:05 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:58:05.305 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:05 smithi102 ceph-mon[41371]: from='osd.1 v1:172.21.15.102:6805/2501991548' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-11T07:58:05.305 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:05 smithi102 ceph-mon[41371]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T07:58:05.305 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:05 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:58:05.305 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:05 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:58:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:05 smithi146 ceph-mon[33158]: from='osd.1 v1:172.21.15.102:6805/2501991548' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-11T07:58:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:05 smithi146 ceph-mon[33158]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T07:58:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:05 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:58:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:05 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:58:06.314 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:06 smithi102 ceph-mon[29849]: purged_snaps scrub starts 2024-09-11T07:58:06.314 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:06 smithi102 ceph-mon[29849]: purged_snaps scrub ok 2024-09-11T07:58:06.314 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:06 smithi102 ceph-mon[29849]: osd.1 v1:172.21.15.102:6805/2501991548 boot 2024-09-11T07:58:06.314 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:06 smithi102 ceph-mon[29849]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T07:58:06.314 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:06 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:58:06.314 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:06 smithi102 ceph-mon[29849]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:58:06.314 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:06 smithi102 ceph-mon[41371]: purged_snaps scrub starts 2024-09-11T07:58:06.315 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:06 smithi102 ceph-mon[41371]: purged_snaps scrub ok 2024-09-11T07:58:06.315 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:06 smithi102 ceph-mon[41371]: osd.1 v1:172.21.15.102:6805/2501991548 boot 2024-09-11T07:58:06.315 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:06 smithi102 ceph-mon[41371]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T07:58:06.315 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:06 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:58:06.315 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:06 smithi102 ceph-mon[41371]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:58:06.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:06 smithi146 ceph-mon[33158]: purged_snaps scrub starts 2024-09-11T07:58:06.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:06 smithi146 ceph-mon[33158]: purged_snaps scrub ok 2024-09-11T07:58:06.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:06 smithi146 ceph-mon[33158]: osd.1 v1:172.21.15.102:6805/2501991548 boot 2024-09-11T07:58:06.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:06 smithi146 ceph-mon[33158]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T07:58:06.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:06 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:58:06.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:06 smithi146 ceph-mon[33158]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:58:07.302 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-11T07:58:07.302 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:07 smithi102 ceph-mon[29849]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T07:58:07.302 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:07 smithi102 ceph-mon[29849]: Detected new or changed devices on smithi102 2024-09-11T07:58:07.303 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:07 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:07.303 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:07 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:07.303 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:07 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:07.303 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:07 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:07.303 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:07 smithi102 ceph-mon[29849]: Adjusting osd_memory_target on smithi102 to 8032M 2024-09-11T07:58:07.303 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:07 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:07.303 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:07 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:07.303 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:07 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:58:07.303 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:07 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:07.303 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:07 smithi102 ceph-mon[41371]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T07:58:07.303 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:07 smithi102 ceph-mon[41371]: Detected new or changed devices on smithi102 2024-09-11T07:58:07.304 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:07 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:07.304 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:07 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:07.304 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:07 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:07.304 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:07 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:07.304 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:07 smithi102 ceph-mon[41371]: Adjusting osd_memory_target on smithi102 to 8032M 2024-09-11T07:58:07.304 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:07 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:07.304 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:07 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:07.304 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:07 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:58:07.304 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:07 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:07.401 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-11T07:58:07.414 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2024-09-11T07:58:07.414 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2024-09-11T07:58:07.414 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114793 s, 913 MB/s 2024-09-11T07:58:07.414 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2024-09-11T07:58:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:07 smithi146 ceph-mon[33158]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T07:58:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:07 smithi146 ceph-mon[33158]: Detected new or changed devices on smithi102 2024-09-11T07:58:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:07 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:07 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:07 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:07 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:07 smithi146 ceph-mon[33158]: Adjusting osd_memory_target on smithi102 to 8032M 2024-09-11T07:58:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:07 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:07 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:07 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:58:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:07 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:07.996 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_2 2024-09-11T07:58:08.240 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:58:08.361 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:08 smithi102 ceph-mon[29849]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:08.361 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:08 smithi102 ceph-mon[41371]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:08.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:08 smithi146 ceph-mon[33158]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:09.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.832+0000 7f9129c34640 1 -- 172.21.15.102:0/3153180933 >> v1:172.21.15.102:6789/0 conn(0x7f91240ff940 legacy=0x7f9124101d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:58:09.863 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.832+0000 7f9129c34640 1 -- 172.21.15.102:0/3153180933 shutdown_connections 2024-09-11T07:58:09.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.832+0000 7f9129c34640 1 -- 172.21.15.102:0/3153180933 >> 172.21.15.102:0/3153180933 conn(0x7f91240f6c40 msgr2=0x7f91240f9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:58:09.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.832+0000 7f9129c34640 1 -- 172.21.15.102:0/3153180933 shutdown_connections 2024-09-11T07:58:09.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.832+0000 7f9129c34640 1 -- 172.21.15.102:0/3153180933 wait complete. 2024-09-11T07:58:09.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.833+0000 7f9129c34640 1 Processor -- start 2024-09-11T07:58:09.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.833+0000 7f9129c34640 1 -- start start 2024-09-11T07:58:09.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.833+0000 7f9129c34640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9124066a90 con 0x7f91240fb440 2024-09-11T07:58:09.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.833+0000 7f9129c34640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9124118850 con 0x7f91240ff940 2024-09-11T07:58:09.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.833+0000 7f9129c34640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f91241199f0 con 0x7f9124103560 2024-09-11T07:58:09.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.833+0000 7f91237fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f91240fb440 0x7f912410f2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:57882/0 (socket says 172.21.15.102:57882) 2024-09-11T07:58:09.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.833+0000 7f91237fe640 1 -- 172.21.15.102:0/1618625123 learned_addr learned my addr 172.21.15.102:0/1618625123 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:58:09.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.833+0000 7f9123fff640 1 --1- 172.21.15.102:0/1618625123 >> v1:172.21.15.102:6790/0 conn(0x7f9124103560 0x7f9124116120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:54860/0 (socket says 172.21.15.102:54860) 2024-09-11T07:58:09.864 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.833+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3081894926 0 0) 0x7f9124066a90 con 0x7f91240fb440 2024-09-11T07:58:09.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.834+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f910c0034e0 con 0x7f91240fb440 2024-09-11T07:58:09.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.834+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2120416599 0 0) 0x7f91241199f0 con 0x7f9124103560 2024-09-11T07:58:09.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.834+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9124066a90 con 0x7f9124103560 2024-09-11T07:58:09.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.834+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000981728 0 0) 0x7f9124118850 con 0x7f91240ff940 2024-09-11T07:58:09.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.834+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91241199f0 con 0x7f91240ff940 2024-09-11T07:58:09.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.834+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 891068150 0 0) 0x7f910c0034e0 con 0x7f91240fb440 2024-09-11T07:58:09.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.834+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9124118850 con 0x7f91240fb440 2024-09-11T07:58:09.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.834+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2058545639 0 0) 0x7f9124066a90 con 0x7f9124103560 2024-09-11T07:58:09.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.834+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f910c0034e0 con 0x7f9124103560 2024-09-11T07:58:09.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.834+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f9114001d70 con 0x7f91240fb440 2024-09-11T07:58:09.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.834+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f911c001800 con 0x7f9124103560 2024-09-11T07:58:09.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.834+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2644069062 0 0) 0x7f91241199f0 con 0x7f91240ff940 2024-09-11T07:58:09.865 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.834+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9124066a90 con 0x7f91240ff940 2024-09-11T07:58:09.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.834+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f9118001770 con 0x7f91240ff940 2024-09-11T07:58:09.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.834+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3207568644 0 0) 0x7f910c0034e0 con 0x7f9124103560 2024-09-11T07:58:09.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.834+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 >> v1:172.21.15.146:6789/0 conn(0x7f91240ff940 legacy=0x7f91241129f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:58:09.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.834+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 >> v1:172.21.15.102:6789/0 conn(0x7f91240fb440 legacy=0x7f912410f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:58:09.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.834+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912411abf0 con 0x7f9124103560 2024-09-11T07:58:09.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.834+0000 7f9129c34640 1 -- 172.21.15.102:0/1618625123 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91241178a0 con 0x7f9124103560 2024-09-11T07:58:09.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.834+0000 7f9129c34640 1 -- 172.21.15.102:0/1618625123 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9124117cf0 con 0x7f9124103560 2024-09-11T07:58:09.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.834+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f911c001df0 con 0x7f9124103560 2024-09-11T07:58:09.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.835+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f911c005920 con 0x7f9124103560 2024-09-11T07:58:09.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.835+0000 7f9129c34640 1 -- 172.21.15.102:0/1618625123 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9110005040 con 0x7f9124103560 2024-09-11T07:58:09.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.835+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 14) v1 ==== 90690+0+0 (unknown 2220093901 0 0) 0x7f911c005b10 con 0x7f9124103560 2024-09-11T07:58:09.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.836+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(14..14 src has 1..14) v4 ==== 1829+0+0 (unknown 1587602299 0 0) 0x7f911c088890 con 0x7f9124103560 2024-09-11T07:58:09.866 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:09.840+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f911c056a50 con 0x7f9124103560 2024-09-11T07:58:10.003 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:10.002+0000 7f9129c34640 1 -- 172.21.15.102:0/1618625123 --> v1:172.21.15.102:6800/2950771867 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9110000bc0 con 0x7f910c06fbf0 2024-09-11T07:58:10.732 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:10 smithi102 ceph-mon[29849]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:10.732 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:10 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:58:10.732 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:10 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:58:10.732 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:10 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:10.732 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:10 smithi102 ceph-mon[41371]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:10.733 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:10 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:58:10.733 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:10 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:58:10.733 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:10 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:10.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:10 smithi146 ceph-mon[33158]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:10.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:10 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:58:10.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:10 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:58:10.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:10 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:11.772 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:11 smithi102 ceph-mon[29849]: from='client.24170 v1:172.21.15.102:0/1618625123' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:58:11.773 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:11 smithi102 ceph-mon[41371]: from='client.24170 v1:172.21.15.102:0/1618625123' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:58:11.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:11 smithi146 ceph-mon[33158]: from='client.24170 v1:172.21.15.102:0/1618625123' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:58:12.777 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:12 smithi102 ceph-mon[41371]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:12.777 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2871439831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e900567f-3dca-4cde-abbb-d8fa53cec635"}]: dispatch 2024-09-11T07:58:12.777 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2871439831' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e900567f-3dca-4cde-abbb-d8fa53cec635"}]': finished 2024-09-11T07:58:12.777 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:12 smithi102 ceph-mon[41371]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T07:58:12.778 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:12 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:58:12.778 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:12 smithi102 ceph-mon[29849]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:12.778 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2871439831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e900567f-3dca-4cde-abbb-d8fa53cec635"}]: dispatch 2024-09-11T07:58:12.778 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2871439831' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e900567f-3dca-4cde-abbb-d8fa53cec635"}]': finished 2024-09-11T07:58:12.778 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:12 smithi102 ceph-mon[29849]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T07:58:12.778 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:12 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:58:12.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:12 smithi146 ceph-mon[33158]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:12.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2871439831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e900567f-3dca-4cde-abbb-d8fa53cec635"}]: dispatch 2024-09-11T07:58:12.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2871439831' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e900567f-3dca-4cde-abbb-d8fa53cec635"}]': finished 2024-09-11T07:58:12.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:12 smithi146 ceph-mon[33158]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T07:58:12.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:12 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:58:13.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2226176464' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:58:13.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2226176464' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:58:13.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2226176464' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:58:14.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:14 smithi102 ceph-mon[41371]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:14.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:14 smithi102 ceph-mon[29849]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:14.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:14 smithi146 ceph-mon[33158]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:16.868 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:16 smithi102 ceph-mon[29849]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:16.868 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:16 smithi102 ceph-mon[41371]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:16.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:16 smithi146 ceph-mon[33158]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:18.838 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:18 smithi102 ceph-mon[29849]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:18.838 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T07:58:18.838 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:18.838 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:18 smithi102 ceph-mon[41371]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:18.838 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T07:58:18.839 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:18.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:18 smithi146 ceph-mon[33158]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:18.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T07:58:18.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:19.731 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:19 smithi102 ceph-mon[29849]: Deploying daemon osd.2 on smithi102 2024-09-11T07:58:19.731 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:19 smithi102 ceph-mon[41371]: Deploying daemon osd.2 on smithi102 2024-09-11T07:58:19.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:19 smithi146 ceph-mon[33158]: Deploying daemon osd.2 on smithi102 2024-09-11T07:58:20.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:20 smithi102 ceph-mon[29849]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:20.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:20 smithi102 ceph-mon[41371]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:20.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:20 smithi146 ceph-mon[33158]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:22.779 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:22 smithi102 ceph-mon[29849]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:22.779 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:22 smithi102 ceph-mon[41371]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:22.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:22 smithi146 ceph-mon[33158]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:23.758 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:23 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:58:23.758 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:23 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:23.759 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:23 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:23.759 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:23 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:58:23.759 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:23 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:23.759 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:23 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:23.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:23 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:58:23.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:23 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:23.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:23 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:24.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:24 smithi102 ceph-mon[41371]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:24.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:24 smithi102 ceph-mon[29849]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:24.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:24 smithi146 ceph-mon[33158]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:26.422 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:26.421+0000 7f910bfff640 1 -- 172.21.15.102:0/1618625123 <== mgr.14150 v1:172.21.15.102:6800/2950771867 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 3552839608) 0x7f9110000bc0 con 0x7f910c06fbf0 2024-09-11T07:58:26.422 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 2 on host 'smithi102' 2024-09-11T07:58:26.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:26.423+0000 7f9129c34640 1 -- 172.21.15.102:0/1618625123 >> v1:172.21.15.102:6800/2950771867 conn(0x7f910c06fbf0 legacy=0x7f910c0720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:58:26.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:26.423+0000 7f9129c34640 1 -- 172.21.15.102:0/1618625123 >> v1:172.21.15.102:6790/0 conn(0x7f9124103560 legacy=0x7f9124116120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:58:26.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:26.423+0000 7f9129c34640 1 -- 172.21.15.102:0/1618625123 shutdown_connections 2024-09-11T07:58:26.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:26.423+0000 7f9129c34640 1 -- 172.21.15.102:0/1618625123 >> 172.21.15.102:0/1618625123 conn(0x7f91240f6c40 msgr2=0x7f91241039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:58:26.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:26.423+0000 7f9129c34640 1 -- 172.21.15.102:0/1618625123 shutdown_connections 2024-09-11T07:58:26.425 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:26.423+0000 7f9129c34640 1 -- 172.21.15.102:0/1618625123 wait complete. 2024-09-11T07:58:26.967 DEBUG:teuthology.orchestra.run.smithi102:osd.2> sudo journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.2.service 2024-09-11T07:58:26.969 INFO:tasks.cephadm:Deploying osd.3 on smithi102 with /dev/vg_nvme/lv_1... 2024-09-11T07:58:26.970 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-11T07:58:26.990 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:26 smithi102 ceph-mon[29849]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:26.990 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:26.990 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:26.991 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:26.991 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:58:26.991 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:26.991 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:58:26.991 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:26.991 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:26.991 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:26 smithi102 ceph-mon[29849]: from='osd.2 v1:172.21.15.102:6809/3842496257' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:58:26.991 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:26 smithi102 ceph-mon[29849]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:58:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:26 smithi102 ceph-mon[41371]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:26.991 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:26.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:26.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:26.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:58:26.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:26.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:58:26.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:26.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:26.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:26 smithi102 ceph-mon[41371]: from='osd.2 v1:172.21.15.102:6809/3842496257' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:58:26.992 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:26 smithi102 ceph-mon[41371]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:58:27.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:26 smithi146 ceph-mon[33158]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:27.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:27.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:27.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:27.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:58:27.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:27.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:58:27.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:27.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:27.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:26 smithi146 ceph-mon[33158]: from='osd.2 v1:172.21.15.102:6809/3842496257' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:58:27.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:26 smithi146 ceph-mon[33158]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:58:28.104 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:58:28.612 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:28 smithi102 ceph-mon[29849]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:28.612 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:28 smithi102 ceph-mon[29849]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T07:58:28.613 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:28 smithi102 ceph-mon[29849]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T07:58:28.613 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:28 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:58:28.613 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:28 smithi102 ceph-mon[29849]: from='osd.2 v1:172.21.15.102:6809/3842496257' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-11T07:58:28.613 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:28 smithi102 ceph-mon[29849]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-11T07:58:28.614 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:28 smithi102 ceph-mon[41371]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:28.614 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:28 smithi102 ceph-mon[41371]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T07:58:28.614 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:28 smithi102 ceph-mon[41371]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T07:58:28.614 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:28 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:58:28.615 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:28 smithi102 ceph-mon[41371]: from='osd.2 v1:172.21.15.102:6809/3842496257' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-11T07:58:28.615 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:28 smithi102 ceph-mon[41371]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-11T07:58:28.615 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 07:58:28 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-2[58066]: 2024-09-11T07:58:28.434+0000 7f33f57dd640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T07:58:28.615 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 07:58:28 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-2[58066]: 2024-09-11T07:58:28.437+0000 7f33efe13640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:58:28.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:28 smithi146 ceph-mon[33158]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:28.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:28 smithi146 ceph-mon[33158]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T07:58:28.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:28 smithi146 ceph-mon[33158]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T07:58:28.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:28 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:58:28.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:28 smithi146 ceph-mon[33158]: from='osd.2 v1:172.21.15.102:6809/3842496257' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-11T07:58:28.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:28 smithi146 ceph-mon[33158]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-11T07:58:29.493 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:29 smithi102 ceph-mon[29849]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-11T07:58:29.493 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:29 smithi102 ceph-mon[29849]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T07:58:29.493 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:29 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:58:29.493 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:29 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:58:29.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:29 smithi102 ceph-mon[41371]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-11T07:58:29.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:29 smithi102 ceph-mon[41371]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T07:58:29.493 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:29 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:58:29.494 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:29 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:58:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:29 smithi146 ceph-mon[33158]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-11T07:58:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:29 smithi146 ceph-mon[33158]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T07:58:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:29 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:58:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:29 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:58:30.148 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-11T07:58:30.239 INFO:teuthology.orchestra.run.smithi102.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-11T07:58:30.256 INFO:teuthology.orchestra.run.smithi102.stderr: stderr: 10+0 records in 2024-09-11T07:58:30.256 INFO:teuthology.orchestra.run.smithi102.stderr:10+0 records out 2024-09-11T07:58:30.256 INFO:teuthology.orchestra.run.smithi102.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.015196 s, 690 MB/s 2024-09-11T07:58:30.256 INFO:teuthology.orchestra.run.smithi102.stderr:--> Zapping successful for: 2024-09-11T07:58:30.743 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:30 smithi102 ceph-mon[41371]: purged_snaps scrub starts 2024-09-11T07:58:30.743 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:30 smithi102 ceph-mon[41371]: purged_snaps scrub ok 2024-09-11T07:58:30.743 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:30 smithi102 ceph-mon[41371]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:30.743 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:30 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:58:30.743 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:30 smithi102 ceph-mon[41371]: osd.2 v1:172.21.15.102:6809/3842496257 boot 2024-09-11T07:58:30.743 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:30 smithi102 ceph-mon[41371]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T07:58:30.743 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:30 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:58:30.743 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:30 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T07:58:30.743 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:30 smithi102 ceph-mon[29849]: purged_snaps scrub starts 2024-09-11T07:58:30.744 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:30 smithi102 ceph-mon[29849]: purged_snaps scrub ok 2024-09-11T07:58:30.744 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:30 smithi102 ceph-mon[29849]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:30.744 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:30 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:58:30.744 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:30 smithi102 ceph-mon[29849]: osd.2 v1:172.21.15.102:6809/3842496257 boot 2024-09-11T07:58:30.744 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:30 smithi102 ceph-mon[29849]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T07:58:30.744 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:30 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:58:30.744 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:30 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T07:58:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:30 smithi146 ceph-mon[33158]: purged_snaps scrub starts 2024-09-11T07:58:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:30 smithi146 ceph-mon[33158]: purged_snaps scrub ok 2024-09-11T07:58:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:30 smithi146 ceph-mon[33158]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:58:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:30 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:58:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:30 smithi146 ceph-mon[33158]: osd.2 v1:172.21.15.102:6809/3842496257 boot 2024-09-11T07:58:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:30 smithi146 ceph-mon[33158]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T07:58:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:30 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:58:30.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:30 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T07:58:31.402 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_1 2024-09-11T07:58:31.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T07:58:31.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[29849]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T07:58:31.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:58:31.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:31.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:31.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:31.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:31.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:31.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:31.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:31.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:58:31.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:31.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T07:58:31.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[41371]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T07:58:31.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:58:31.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:31.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:31.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:31.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:31.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:31.650 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:31.650 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:31.650 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:58:31.650 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:31 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:31.650 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:58:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:31 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T07:58:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:31 smithi146 ceph-mon[33158]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T07:58:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:31 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:58:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:31 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:31 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:31 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:31 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:31 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:31 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:31.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:31 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:31.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:31 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:58:31.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:31 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:32.840 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 07:58:32 smithi102 sudo[62281]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:58:32.840 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 07:58:32 smithi102 sudo[62281]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:58:32.840 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 07:58:32 smithi102 sudo[62281]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:58:32.840 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 07:58:32 smithi102 sudo[62281]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:58:32.841 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:32 smithi102 ceph-mon[29849]: Detected new or changed devices on smithi102 2024-09-11T07:58:32.841 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:32 smithi102 ceph-mon[29849]: Adjusting osd_memory_target on smithi102 to 5354M 2024-09-11T07:58:32.841 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:32 smithi102 ceph-mon[29849]: pgmap v73: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:32.841 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:32 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:58:32.841 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:32 smithi102 ceph-mon[29849]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T07:58:32.841 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:32 smithi102 ceph-mon[29849]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:58:32.841 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:32 smithi102 ceph-mon[41371]: Detected new or changed devices on smithi102 2024-09-11T07:58:32.842 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:32 smithi102 ceph-mon[41371]: Adjusting osd_memory_target on smithi102 to 5354M 2024-09-11T07:58:32.842 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:32 smithi102 ceph-mon[41371]: pgmap v73: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:32.842 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:32 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:58:32.842 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:32 smithi102 ceph-mon[41371]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T07:58:32.842 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:32 smithi102 ceph-mon[41371]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:58:32.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:32 smithi146 ceph-mon[33158]: Detected new or changed devices on smithi102 2024-09-11T07:58:32.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:32 smithi146 ceph-mon[33158]: Adjusting osd_memory_target on smithi102 to 5354M 2024-09-11T07:58:32.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:32 smithi146 ceph-mon[33158]: pgmap v73: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:32.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:32 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:58:32.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:32 smithi146 ceph-mon[33158]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T07:58:32.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:32 smithi146 ceph-mon[33158]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:58:33.107 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 07:58:32 smithi102 sudo[62284]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:58:33.108 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 07:58:32 smithi102 sudo[62284]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:58:33.108 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 07:58:32 smithi102 sudo[62284]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:58:33.108 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 07:58:32 smithi102 sudo[62284]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:58:33.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.304+0000 7fdc01fee640 1 -- 172.21.15.102:0/86356950 >> v1:172.21.15.102:6789/0 conn(0x7fdbfc0fb480 legacy=0x7fdbfc0fb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:58:33.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.304+0000 7fdc01fee640 1 -- 172.21.15.102:0/86356950 shutdown_connections 2024-09-11T07:58:33.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.304+0000 7fdc01fee640 1 -- 172.21.15.102:0/86356950 >> 172.21.15.102:0/86356950 conn(0x7fdbfc0f6c20 msgr2=0x7fdbfc0f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:58:33.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.304+0000 7fdc01fee640 1 -- 172.21.15.102:0/86356950 shutdown_connections 2024-09-11T07:58:33.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.304+0000 7fdc01fee640 1 -- 172.21.15.102:0/86356950 wait complete. 2024-09-11T07:58:33.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.304+0000 7fdc01fee640 1 Processor -- start 2024-09-11T07:58:33.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.304+0000 7fdc01fee640 1 -- start start 2024-09-11T07:58:33.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdc01fee640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdbfc1175f0 con 0x7fdbfc1035a0 2024-09-11T07:58:33.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdc01fee640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdbfc1187f0 con 0x7fdbfc0fb480 2024-09-11T07:58:33.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdc01fee640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdbfc1199f0 con 0x7fdbfc0ff980 2024-09-11T07:58:33.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdbf3fff640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7fdbfc0ff980 0x7fdbfc112340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:44378/0 (socket says 172.21.15.102:44378) 2024-09-11T07:58:33.322 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdbf3fff640 1 -- 172.21.15.102:0/586102101 learned_addr learned my addr 172.21.15.102:0/586102101 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T07:58:33.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 991164167 0 0) 0x7fdbfc1199f0 con 0x7fdbfc0ff980 2024-09-11T07:58:33.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbe8003120 con 0x7fdbfc0ff980 2024-09-11T07:58:33.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2191302515 0 0) 0x7fdbfc1175f0 con 0x7fdbfc1035a0 2024-09-11T07:58:33.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbfc1199f0 con 0x7fdbfc1035a0 2024-09-11T07:58:33.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 982014764 0 0) 0x7fdbfc1187f0 con 0x7fdbfc0fb480 2024-09-11T07:58:33.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbfc1175f0 con 0x7fdbfc0fb480 2024-09-11T07:58:33.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4087179030 0 0) 0x7fdbe8003120 con 0x7fdbfc0ff980 2024-09-11T07:58:33.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdbfc1187f0 con 0x7fdbfc0ff980 2024-09-11T07:58:33.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fdbe40019f0 con 0x7fdbfc0ff980 2024-09-11T07:58:33.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2574683188 0 0) 0x7fdbfc1199f0 con 0x7fdbfc1035a0 2024-09-11T07:58:33.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdbe8003120 con 0x7fdbfc1035a0 2024-09-11T07:58:33.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1641152241 0 0) 0x7fdbfc1175f0 con 0x7fdbfc0fb480 2024-09-11T07:58:33.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdbfc1199f0 con 0x7fdbfc0fb480 2024-09-11T07:58:33.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fdbf4001bc0 con 0x7fdbfc1035a0 2024-09-11T07:58:33.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fdbec0019c0 con 0x7fdbfc0fb480 2024-09-11T07:58:33.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 61202039 0 0) 0x7fdbfc1187f0 con 0x7fdbfc0ff980 2024-09-11T07:58:33.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 >> v1:172.21.15.146:6789/0 conn(0x7fdbfc0fb480 legacy=0x7fdbfc065860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:58:33.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 >> v1:172.21.15.102:6789/0 conn(0x7fdbfc1035a0 legacy=0x7fdbfc115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:58:33.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.305+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbfc11abf0 con 0x7fdbfc0ff980 2024-09-11T07:58:33.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.306+0000 7fdc01fee640 1 -- 172.21.15.102:0/586102101 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbfc1177e0 con 0x7fdbfc0ff980 2024-09-11T07:58:33.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.306+0000 7fdc01fee640 1 -- 172.21.15.102:0/586102101 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbfc117cb0 con 0x7fdbfc0ff980 2024-09-11T07:58:33.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.306+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7fdbe4002500 con 0x7fdbfc0ff980 2024-09-11T07:58:33.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.306+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fdbe4005b20 con 0x7fdbfc0ff980 2024-09-11T07:58:33.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.307+0000 7fdc01fee640 1 -- 172.21.15.102:0/586102101 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbdc005040 con 0x7fdbfc0ff980 2024-09-11T07:58:33.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.307+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 14) v1 ==== 90690+0+0 (unknown 2220093901 0 0) 0x7fdbe40020b0 con 0x7fdbfc0ff980 2024-09-11T07:58:33.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.308+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(21..21 src has 1..21) v4 ==== 2507+0+0 (unknown 1953984833 0 0) 0x7fdbe4088e10 con 0x7fdbfc0ff980 2024-09-11T07:58:33.324 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.312+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fdbe4056a80 con 0x7fdbfc0ff980 2024-09-11T07:58:33.380 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 07:58:33 smithi102 sudo[62321]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:58:33.381 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 07:58:33 smithi102 sudo[62321]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:58:33.381 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 07:58:33 smithi102 sudo[62321]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:58:33.381 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 07:58:33 smithi102 sudo[62321]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:58:33.483 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:33.482+0000 7fdc01fee640 1 -- 172.21.15.102:0/586102101 --> v1:172.21.15.102:6800/2950771867 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbdc000bc0 con 0x7fdbe806fcd0 2024-09-11T07:58:33.637 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 07:58:33 smithi102 sudo[62355]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:58:33.637 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 07:58:33 smithi102 sudo[62355]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:58:33.637 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 07:58:33 smithi102 sudo[62355]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:58:33.637 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 07:58:33 smithi102 sudo[62355]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:58:33.895 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 07:58:33 smithi102 sudo[62425]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:58:33.896 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 07:58:33 smithi102 sudo[62425]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:58:33.896 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 07:58:33 smithi102 sudo[62425]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:58:33.896 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 07:58:33 smithi102 sudo[62425]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:58:33.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:33 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:33.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:33 smithi102 ceph-mon[29849]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T07:58:33.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:33 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:58:33.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:33 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:58:33.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:33 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:33.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:33 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:33.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:33 smithi102 ceph-mon[41371]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T07:58:33.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:33 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:58:33.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:33 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:58:33.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:33 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:34.167 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 07:58:33 smithi102 sudo[62482]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:58:34.167 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 07:58:33 smithi102 sudo[62482]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:58:34.167 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 07:58:33 smithi102 sudo[62482]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:58:34.167 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 07:58:33 smithi102 sudo[62482]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:58:34.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:33 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:34.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:33 smithi146 ceph-mon[33158]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T07:58:34.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:33 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:58:34.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:33 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:58:34.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:33 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:34.424 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:34 smithi102 sudo[62522]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:58:34.425 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:34 smithi102 sudo[62522]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:58:34.425 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:34 smithi102 sudo[62522]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:58:34.425 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:34 smithi102 sudo[62522]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:58:34.803 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:34 smithi102 sudo[62525]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:58:34.803 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:34 smithi102 sudo[62525]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:58:34.803 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:34 smithi102 sudo[62525]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:58:34.803 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:34 smithi102 sudo[62525]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:58:34.925 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:34 smithi146 sudo[35889]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:58:34.925 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:34 smithi146 sudo[35889]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:58:34.926 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:34 smithi146 sudo[35889]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:58:34.926 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:34 smithi146 sudo[35889]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:58:34.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:34.934+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 <== mon.2 v1:172.21.15.102:6790/0 10 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 1029906827 0 0) 0x7fdbe4052630 con 0x7fdbfc0ff980 2024-09-11T07:58:35.083 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[29849]: pgmap v76: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:35.084 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[29849]: from='client.24203 v1:172.21.15.102:0/586102101' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:58:35.084 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:58:35.084 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:58:35.084 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:58:35.084 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[29849]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:58:35.084 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[29849]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:58:35.084 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[29849]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:58:35.084 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[29849]: Cluster is now healthy 2024-09-11T07:58:35.084 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:58:35.084 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:58:35.084 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:58:35.085 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[41371]: pgmap v76: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:35.085 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[41371]: from='client.24203 v1:172.21.15.102:0/586102101' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:58:35.085 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:58:35.085 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:58:35.085 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:58:35.085 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[41371]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:58:35.085 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[41371]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:58:35.085 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[41371]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:58:35.085 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[41371]: Cluster is now healthy 2024-09-11T07:58:35.085 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:58:35.085 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:58:35.086 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:34 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:58:35.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:34 smithi146 sudo[35892]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:58:35.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:34 smithi146 sudo[35892]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:58:35.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:34 smithi146 sudo[35892]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:58:35.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:34 smithi146 sudo[35892]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:58:35.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:34 smithi146 ceph-mon[33158]: pgmap v76: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:35.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:34 smithi146 ceph-mon[33158]: from='client.24203 v1:172.21.15.102:0/586102101' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:58:35.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:34 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:58:35.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:34 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:58:35.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:34 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:58:35.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:34 smithi146 ceph-mon[33158]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:58:35.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:34 smithi146 ceph-mon[33158]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:58:35.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:34 smithi146 ceph-mon[33158]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:58:35.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:34 smithi146 ceph-mon[33158]: Cluster is now healthy 2024-09-11T07:58:35.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:34 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:58:35.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:34 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:58:35.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:34 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:58:35.348 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:35 smithi102 sudo[62576]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:58:35.349 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:35 smithi102 sudo[62576]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:58:35.349 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:35 smithi102 sudo[62576]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:58:35.349 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:35 smithi102 sudo[62576]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:58:35.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:35 smithi102 sudo[62602]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:58:35.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:35 smithi102 sudo[62602]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:58:35.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:35 smithi102 sudo[62602]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:58:35.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:35 smithi102 sudo[62602]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:58:36.318 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[29849]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:58:36.319 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[29849]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:58:36.319 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[29849]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T07:58:36.319 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:36.319 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:58:36.319 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:58:36.319 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[29849]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:58:36.319 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:58:36.319 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[29849]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:58:36.319 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2361407903' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0d77f9c-0e1b-45b5-8b9b-e84097814fb1"}]: dispatch 2024-09-11T07:58:36.319 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[29849]: from='client.24212 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0d77f9c-0e1b-45b5-8b9b-e84097814fb1"}]: dispatch 2024-09-11T07:58:36.319 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[29849]: from='client.24212 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0d77f9c-0e1b-45b5-8b9b-e84097814fb1"}]': finished 2024-09-11T07:58:36.319 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[29849]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T07:58:36.319 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:58:36.320 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[41371]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:58:36.320 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[41371]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:58:36.320 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[41371]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T07:58:36.320 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:36.320 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:58:36.320 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:58:36.320 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[41371]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:58:36.320 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:58:36.320 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[41371]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:58:36.321 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2361407903' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0d77f9c-0e1b-45b5-8b9b-e84097814fb1"}]: dispatch 2024-09-11T07:58:36.321 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[41371]: from='client.24212 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0d77f9c-0e1b-45b5-8b9b-e84097814fb1"}]: dispatch 2024-09-11T07:58:36.321 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[41371]: from='client.24212 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0d77f9c-0e1b-45b5-8b9b-e84097814fb1"}]': finished 2024-09-11T07:58:36.321 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[41371]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T07:58:36.321 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:35 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:58:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:35 smithi146 ceph-mon[33158]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:58:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:35 smithi146 ceph-mon[33158]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:58:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:35 smithi146 ceph-mon[33158]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T07:58:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:35 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:35 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:58:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:35 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:58:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:35 smithi146 ceph-mon[33158]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:58:36.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:35 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:58:36.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:35 smithi146 ceph-mon[33158]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:58:36.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2361407903' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0d77f9c-0e1b-45b5-8b9b-e84097814fb1"}]: dispatch 2024-09-11T07:58:36.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:35 smithi146 ceph-mon[33158]: from='client.24212 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c0d77f9c-0e1b-45b5-8b9b-e84097814fb1"}]: dispatch 2024-09-11T07:58:36.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:35 smithi146 ceph-mon[33158]: from='client.24212 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0d77f9c-0e1b-45b5-8b9b-e84097814fb1"}]': finished 2024-09-11T07:58:36.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:35 smithi146 ceph-mon[33158]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T07:58:36.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:35 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:58:37.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:36 smithi102 ceph-mon[29849]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:37.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:36 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2366951170' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:58:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:36 smithi102 ceph-mon[41371]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:36 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2366951170' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:58:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:36 smithi146 ceph-mon[33158]: pgmap v77: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:36 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2366951170' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:58:39.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:38 smithi102 ceph-mon[29849]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:39.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:38 smithi102 ceph-mon[41371]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:39.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:38 smithi146 ceph-mon[33158]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:40 smithi102 ceph-mon[29849]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:41.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:40 smithi102 ceph-mon[41371]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:40 smithi146 ceph-mon[33158]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:42.220 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:41 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T07:58:42.220 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:41 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:42.221 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:41 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T07:58:42.221 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:41 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:42.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:41 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T07:58:42.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:41 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:43.081 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:42 smithi102 ceph-mon[29849]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:43.082 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:42 smithi102 ceph-mon[29849]: Deploying daemon osd.3 on smithi102 2024-09-11T07:58:43.082 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:42 smithi102 ceph-mon[41371]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:43.082 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:42 smithi102 ceph-mon[41371]: Deploying daemon osd.3 on smithi102 2024-09-11T07:58:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:42 smithi146 ceph-mon[33158]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:42 smithi146 ceph-mon[33158]: Deploying daemon osd.3 on smithi102 2024-09-11T07:58:45.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:44 smithi102 ceph-mon[29849]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:45.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:44 smithi102 ceph-mon[41371]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:44 smithi146 ceph-mon[33158]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:47.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:46 smithi102 ceph-mon[29849]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:47.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:46 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:58:47.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:46 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:47.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:46 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:47.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:46 smithi102 ceph-mon[41371]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:47.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:46 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:58:47.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:46 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:47.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:46 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:47.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:46 smithi146 ceph-mon[33158]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:47.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:46 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:58:47.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:46 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:47.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:46 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:49.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:48 smithi102 ceph-mon[29849]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:49.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:48 smithi102 ceph-mon[41371]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:49.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:48 smithi146 ceph-mon[33158]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:49.883 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:49.882+0000 7fdbf8ff9640 1 -- 172.21.15.102:0/586102101 <== mgr.14150 v1:172.21.15.102:6800/2950771867 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 2412898572) 0x7fdbdc000bc0 con 0x7fdbe806fcd0 2024-09-11T07:58:49.884 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 3 on host 'smithi102' 2024-09-11T07:58:49.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:49.884+0000 7fdc01fee640 1 -- 172.21.15.102:0/586102101 >> v1:172.21.15.102:6800/2950771867 conn(0x7fdbe806fcd0 legacy=0x7fdbe8072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:58:49.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:49.884+0000 7fdc01fee640 1 -- 172.21.15.102:0/586102101 >> v1:172.21.15.102:6790/0 conn(0x7fdbfc0ff980 legacy=0x7fdbfc112340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:58:49.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:49.884+0000 7fdc01fee640 1 -- 172.21.15.102:0/586102101 shutdown_connections 2024-09-11T07:58:49.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:49.884+0000 7fdc01fee640 1 -- 172.21.15.102:0/586102101 >> 172.21.15.102:0/586102101 conn(0x7fdbfc0f6c20 msgr2=0x7fdbfc105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:58:49.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:49.885+0000 7fdc01fee640 1 -- 172.21.15.102:0/586102101 shutdown_connections 2024-09-11T07:58:49.886 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T07:58:49.885+0000 7fdc01fee640 1 -- 172.21.15.102:0/586102101 wait complete. 2024-09-11T07:58:50.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:50 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:50.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:50 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:50.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:50 smithi146 ceph-mon[33158]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:50.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:50 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:50.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:50 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:58:50.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:50 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:50.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:50 smithi146 ceph-mon[33158]: from='osd.3 v1:172.21.15.102:6813/426874071' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:58:50.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:50 smithi146 ceph-mon[33158]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:58:50.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:50 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:58:50.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:50 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:50.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:50 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:50.504 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:50 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:50.504 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:50 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:50.504 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:50 smithi102 ceph-mon[41371]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:50.504 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:50 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:50.504 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:50 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:58:50.504 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:50 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:50.505 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:50 smithi102 ceph-mon[41371]: from='osd.3 v1:172.21.15.102:6813/426874071' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:58:50.505 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:50 smithi102 ceph-mon[41371]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:58:50.505 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:50 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:58:50.505 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:50 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:50.505 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:50 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:50.505 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:50 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:50.505 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:50 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:50.505 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:50 smithi102 ceph-mon[29849]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:50.505 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:50 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:50.505 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:50 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:58:50.505 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:50 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:50.506 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:50 smithi102 ceph-mon[29849]: from='osd.3 v1:172.21.15.102:6813/426874071' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:58:50.506 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:50 smithi102 ceph-mon[29849]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:58:50.506 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:50 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:58:50.506 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:50 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:50.506 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:50 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:50.527 DEBUG:teuthology.orchestra.run.smithi102:osd.3> sudo journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.3.service 2024-09-11T07:58:50.531 INFO:tasks.cephadm:Deploying osd.4 on smithi146 with /dev/vg_nvme/lv_4... 2024-09-11T07:58:50.531 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-11T07:58:50.782 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:58:51.644 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:51 smithi102 ceph-mon[29849]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T07:58:51.644 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:51 smithi102 ceph-mon[29849]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T07:58:51.644 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:51 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:58:51.644 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:51 smithi102 ceph-mon[29849]: from='osd.3 v1:172.21.15.102:6813/426874071' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-11T07:58:51.644 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:51 smithi102 ceph-mon[29849]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-11T07:58:51.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:51 smithi102 ceph-mon[41371]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T07:58:51.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:51 smithi102 ceph-mon[41371]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T07:58:51.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:51 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:58:51.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:51 smithi102 ceph-mon[41371]: from='osd.3 v1:172.21.15.102:6813/426874071' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-11T07:58:51.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:51 smithi102 ceph-mon[41371]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-11T07:58:51.645 INFO:journalctl@ceph.osd.3.smithi102.stdout:Sep 11 07:58:51 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-3[64421]: 2024-09-11T07:58:51.592+0000 7f3a03d92640 -1 osd.3 0 waiting for initial osdmap 2024-09-11T07:58:51.645 INFO:journalctl@ceph.osd.3.smithi102.stdout:Sep 11 07:58:51 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-3[64421]: 2024-09-11T07:58:51.596+0000 7f39fe3c8640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:58:51.887 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:51 smithi146 ceph-mon[33158]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T07:58:51.887 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:51 smithi146 ceph-mon[33158]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T07:58:51.888 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:51 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:58:51.888 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:51 smithi146 ceph-mon[33158]: from='osd.3 v1:172.21.15.102:6813/426874071' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-11T07:58:51.888 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:51 smithi146 ceph-mon[33158]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-11T07:58:52.395 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-11T07:58:52.504 INFO:teuthology.orchestra.run.smithi146.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-11T07:58:52.517 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10+0 records in 2024-09-11T07:58:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:10+0 records out 2024-09-11T07:58:52.518 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115534 s, 908 MB/s 2024-09-11T07:58:52.518 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping successful for: 2024-09-11T07:58:52.775 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:52 smithi146 ceph-mon[33158]: purged_snaps scrub starts 2024-09-11T07:58:52.775 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:52 smithi146 ceph-mon[33158]: purged_snaps scrub ok 2024-09-11T07:58:52.775 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:52 smithi146 ceph-mon[33158]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:52.775 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:52 smithi146 ceph-mon[33158]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-11T07:58:52.776 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:52 smithi146 ceph-mon[33158]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T07:58:52.776 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:52 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:58:52.776 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:52 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:58:52.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:52 smithi102 ceph-mon[29849]: purged_snaps scrub starts 2024-09-11T07:58:52.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:52 smithi102 ceph-mon[29849]: purged_snaps scrub ok 2024-09-11T07:58:52.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:52 smithi102 ceph-mon[29849]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:52.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:52 smithi102 ceph-mon[29849]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-11T07:58:52.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:52 smithi102 ceph-mon[29849]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T07:58:52.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:52 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:58:52.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:52 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:58:52.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:52 smithi102 ceph-mon[41371]: purged_snaps scrub starts 2024-09-11T07:58:52.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:52 smithi102 ceph-mon[41371]: purged_snaps scrub ok 2024-09-11T07:58:52.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:52 smithi102 ceph-mon[41371]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:52.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:52 smithi102 ceph-mon[41371]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-11T07:58:52.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:52 smithi102 ceph-mon[41371]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T07:58:52.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:52 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:58:52.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:52 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:58:53.100 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi146:vg_nvme/lv_4 2024-09-11T07:58:53.342 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:58:53.728 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:53 smithi146 ceph-mon[33158]: osd.3 v1:172.21.15.102:6813/426874071 boot 2024-09-11T07:58:53.729 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:53 smithi146 ceph-mon[33158]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T07:58:53.729 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:53 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:58:53.729 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:53 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:53.729 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:53 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:53.729 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:53 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:53.729 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:53 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:53.729 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:53 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:53.729 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:53 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:53.729 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:53 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:53.729 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:53 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:53.730 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:53 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:58:53.730 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:53 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:53.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[29849]: osd.3 v1:172.21.15.102:6813/426874071 boot 2024-09-11T07:58:53.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[29849]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T07:58:53.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:58:53.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:53.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:53.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:53.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:53.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:53.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:53.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:53.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:53.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:58:53.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:53.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[41371]: osd.3 v1:172.21.15.102:6813/426874071 boot 2024-09-11T07:58:53.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[41371]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T07:58:53.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:58:53.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:53.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:53.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:53.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:53.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:53.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:58:53.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:53.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:53.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:58:53.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:53 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:58:54.813 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:54 smithi146 ceph-mon[33158]: Detected new or changed devices on smithi102 2024-09-11T07:58:54.813 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:54 smithi146 ceph-mon[33158]: Adjusting osd_memory_target on smithi102 to 4016M 2024-09-11T07:58:54.813 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:54 smithi146 ceph-mon[33158]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:54.814 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:54 smithi146 ceph-mon[33158]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T07:58:54.833 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.811+0000 7f8635acb640 1 -- 172.21.15.146:0/3162085394 >> v1:172.21.15.146:6789/0 conn(0x7f86300fb480 legacy=0x7f86300fb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:58:54.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.811+0000 7f8635acb640 1 -- 172.21.15.146:0/3162085394 shutdown_connections 2024-09-11T07:58:54.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.811+0000 7f8635acb640 1 -- 172.21.15.146:0/3162085394 >> 172.21.15.146:0/3162085394 conn(0x7f86300f6c20 msgr2=0x7f86300f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:58:54.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.811+0000 7f8635acb640 1 -- 172.21.15.146:0/3162085394 shutdown_connections 2024-09-11T07:58:54.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.811+0000 7f8635acb640 1 -- 172.21.15.146:0/3162085394 wait complete. 2024-09-11T07:58:54.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.812+0000 7f8635acb640 1 Processor -- start 2024-09-11T07:58:54.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.812+0000 7f8635acb640 1 -- start start 2024-09-11T07:58:54.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.812+0000 7f8635acb640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f863010d1d0 con 0x7f86300ff980 2024-09-11T07:58:54.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.812+0000 7f8635acb640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f86301189d0 con 0x7f86300fb480 2024-09-11T07:58:54.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.812+0000 7f8635acb640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8630119b70 con 0x7f86301035a0 2024-09-11T07:58:54.834 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.812+0000 7f862f7fe640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f86300fb480 0x7f863010c640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:55582/0 (socket says 172.21.15.146:55582) 2024-09-11T07:58:54.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.812+0000 7f862f7fe640 1 -- 172.21.15.146:0/1449041707 learned_addr learned my addr 172.21.15.146:0/1449041707 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-11T07:58:54.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.812+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548700691 0 0) 0x7f86301189d0 con 0x7f86300fb480 2024-09-11T07:58:54.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.812+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f861c003120 con 0x7f86300fb480 2024-09-11T07:58:54.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.812+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2992823266 0 0) 0x7f863010d1d0 con 0x7f86300ff980 2024-09-11T07:58:54.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.812+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86301189d0 con 0x7f86300ff980 2024-09-11T07:58:54.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.812+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2479835479 0 0) 0x7f8630119b70 con 0x7f86301035a0 2024-09-11T07:58:54.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.813+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f863010d1d0 con 0x7f86301035a0 2024-09-11T07:58:54.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.813+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1148224737 0 0) 0x7f861c003120 con 0x7f86300fb480 2024-09-11T07:58:54.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.813+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8630119b70 con 0x7f86300fb480 2024-09-11T07:58:54.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.813+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f8620000fe0 con 0x7f86300fb480 2024-09-11T07:58:54.835 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.813+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2960900477 0 0) 0x7f86301189d0 con 0x7f86300ff980 2024-09-11T07:58:54.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.813+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f861c003120 con 0x7f86300ff980 2024-09-11T07:58:54.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.813+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f86180018c0 con 0x7f86300ff980 2024-09-11T07:58:54.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.813+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4133338924 0 0) 0x7f863010d1d0 con 0x7f86301035a0 2024-09-11T07:58:54.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.813+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f86301189d0 con 0x7f86301035a0 2024-09-11T07:58:54.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.813+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 <== mon.1 v1:172.21.15.146:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1934794997 0 0) 0x7f8630119b70 con 0x7f86300fb480 2024-09-11T07:58:54.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.813+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 >> v1:172.21.15.102:6790/0 conn(0x7f86301035a0 legacy=0x7f86301162a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:58:54.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.813+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 >> v1:172.21.15.102:6789/0 conn(0x7f86300ff980 legacy=0x7f8630112b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:58:54.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.813+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f863011ad50 con 0x7f86300fb480 2024-09-11T07:58:54.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.813+0000 7f8635acb640 1 -- 172.21.15.146:0/1449041707 --> v1:172.21.15.146:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8630117a20 con 0x7f86300fb480 2024-09-11T07:58:54.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.813+0000 7f8635acb640 1 -- 172.21.15.146:0/1449041707 --> v1:172.21.15.146:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8630117ef0 con 0x7f86300fb480 2024-09-11T07:58:54.836 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.813+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 <== mon.1 v1:172.21.15.146:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f8620002c90 con 0x7f86300fb480 2024-09-11T07:58:54.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.813+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 <== mon.1 v1:172.21.15.146:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f8620003730 con 0x7f86300fb480 2024-09-11T07:58:54.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.814+0000 7f8635acb640 1 -- 172.21.15.146:0/1449041707 --> v1:172.21.15.146:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8610005040 con 0x7f86300fb480 2024-09-11T07:58:54.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.814+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 <== mon.1 v1:172.21.15.146:6789/0 7 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 1029906827 0 0) 0x7f8620002160 con 0x7f86300fb480 2024-09-11T07:58:54.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.815+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 <== mon.1 v1:172.21.15.146:6789/0 8 ==== osd_map(26..26 src has 1..26) v4 ==== 2799+0+0 (unknown 3804451695 0 0) 0x7f86200893a0 con 0x7f86300fb480 2024-09-11T07:58:54.837 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.819+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 <== mon.1 v1:172.21.15.146:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f8620056e10 con 0x7f86300fb480 2024-09-11T07:58:54.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:54 smithi102 ceph-mon[29849]: Detected new or changed devices on smithi102 2024-09-11T07:58:54.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:54 smithi102 ceph-mon[29849]: Adjusting osd_memory_target on smithi102 to 4016M 2024-09-11T07:58:54.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:54 smithi102 ceph-mon[29849]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:54.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:54 smithi102 ceph-mon[29849]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T07:58:54.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:54 smithi102 ceph-mon[41371]: Detected new or changed devices on smithi102 2024-09-11T07:58:54.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:54 smithi102 ceph-mon[41371]: Adjusting osd_memory_target on smithi102 to 4016M 2024-09-11T07:58:54.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:54 smithi102 ceph-mon[41371]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:58:54.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:54 smithi102 ceph-mon[41371]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T07:58:54.983 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:58:54.981+0000 7f8635acb640 1 -- 172.21.15.146:0/1449041707 --> v1:172.21.15.102:6800/2950771867 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f8610000bc0 con 0x7f861c06fd60 2024-09-11T07:58:55.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:55 smithi146 ceph-mon[33158]: from='client.24232 v1:172.21.15.146:0/1449041707' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:58:55.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:55 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:58:55.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:55 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:58:55.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:55 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:55.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:55 smithi102 ceph-mon[29849]: from='client.24232 v1:172.21.15.146:0/1449041707' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:58:55.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:55 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:58:55.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:55 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:58:55.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:55 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:55.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:55 smithi102 ceph-mon[41371]: from='client.24232 v1:172.21.15.146:0/1449041707' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:58:55.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:55 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:58:55.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:55 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:58:55.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:55 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:58:56.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:56 smithi102 ceph-mon[29849]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:58:56.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:56 smithi102 ceph-mon[41371]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:58:56.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:56 smithi146 ceph-mon[33158]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:58:57.776 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:57 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.146:0/4099037999' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31d92bc5-10be-47d0-b91c-5e5496c25ebd"}]: dispatch 2024-09-11T07:58:57.776 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:57 smithi146 ceph-mon[33158]: from='client.24238 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31d92bc5-10be-47d0-b91c-5e5496c25ebd"}]: dispatch 2024-09-11T07:58:57.776 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:57 smithi146 ceph-mon[33158]: from='client.24238 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31d92bc5-10be-47d0-b91c-5e5496c25ebd"}]': finished 2024-09-11T07:58:57.776 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:57 smithi146 ceph-mon[33158]: osdmap e27: 5 total, 4 up, 5 in 2024-09-11T07:58:57.776 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:57 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:58:57.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:57 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.146:0/4099037999' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31d92bc5-10be-47d0-b91c-5e5496c25ebd"}]: dispatch 2024-09-11T07:58:57.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:57 smithi102 ceph-mon[29849]: from='client.24238 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31d92bc5-10be-47d0-b91c-5e5496c25ebd"}]: dispatch 2024-09-11T07:58:57.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:57 smithi102 ceph-mon[29849]: from='client.24238 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31d92bc5-10be-47d0-b91c-5e5496c25ebd"}]': finished 2024-09-11T07:58:57.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:57 smithi102 ceph-mon[29849]: osdmap e27: 5 total, 4 up, 5 in 2024-09-11T07:58:57.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:57 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:58:57.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:57 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.146:0/4099037999' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31d92bc5-10be-47d0-b91c-5e5496c25ebd"}]: dispatch 2024-09-11T07:58:57.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:57 smithi102 ceph-mon[41371]: from='client.24238 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "31d92bc5-10be-47d0-b91c-5e5496c25ebd"}]: dispatch 2024-09-11T07:58:57.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:57 smithi102 ceph-mon[41371]: from='client.24238 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "31d92bc5-10be-47d0-b91c-5e5496c25ebd"}]': finished 2024-09-11T07:58:57.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:57 smithi102 ceph-mon[41371]: osdmap e27: 5 total, 4 up, 5 in 2024-09-11T07:58:57.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:57 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:58:58.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:58 smithi102 ceph-mon[29849]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:58:58.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:58:58 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.146:0/163968876' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:58:58.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:58 smithi102 ceph-mon[41371]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:58:58.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:58:58 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.146:0/163968876' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:58:58.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:58 smithi146 ceph-mon[33158]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:58:58.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:58:58 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.146:0/163968876' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:59:00.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:00 smithi102 ceph-mon[29849]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:00.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:00 smithi102 ceph-mon[41371]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:00.923 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:00 smithi146 ceph-mon[33158]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:02.775 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:02 smithi146 ceph-mon[33158]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:02.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:02 smithi102 ceph-mon[29849]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:02.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:02 smithi102 ceph-mon[41371]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:03.736 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:03 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T07:59:03.736 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:03 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:03.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:03 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T07:59:03.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:03 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:03.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:03 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T07:59:03.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:03 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:04.751 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:04 smithi146 ceph-mon[33158]: Deploying daemon osd.4 on smithi146 2024-09-11T07:59:04.751 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:04 smithi146 ceph-mon[33158]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:04.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:04 smithi102 ceph-mon[41371]: Deploying daemon osd.4 on smithi146 2024-09-11T07:59:04.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:04 smithi102 ceph-mon[41371]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:04.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:04 smithi102 ceph-mon[29849]: Deploying daemon osd.4 on smithi146 2024-09-11T07:59:04.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:04 smithi102 ceph-mon[29849]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:06.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:06 smithi102 ceph-mon[41371]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:06.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:06 smithi102 ceph-mon[29849]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:06.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:06 smithi146 ceph-mon[33158]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:08.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:08 smithi102 ceph-mon[29849]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:08.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:08 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:08.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:08 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:08.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:08 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:08.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:08 smithi102 ceph-mon[41371]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:08.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:08 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:08.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:08 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:08.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:08 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:08.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:08 smithi146 ceph-mon[33158]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:08.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:08 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:08.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:08 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:08.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:08 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:10.762 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:10.760+0000 7f8627fff640 1 -- 172.21.15.146:0/1449041707 <== mgr.14150 v1:172.21.15.102:6800/2950771867 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 4222124824) 0x7f8610000bc0 con 0x7f861c06fd60 2024-09-11T07:59:10.762 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 4 on host 'smithi146' 2024-09-11T07:59:10.762 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:10 smithi146 ceph-mon[33158]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:10.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:10.762+0000 7f8635acb640 1 -- 172.21.15.146:0/1449041707 >> v1:172.21.15.102:6800/2950771867 conn(0x7f861c06fd60 legacy=0x7f861c072210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:59:10.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:10.762+0000 7f8635acb640 1 -- 172.21.15.146:0/1449041707 >> v1:172.21.15.146:6789/0 conn(0x7f86300fb480 legacy=0x7f863010c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:59:10.764 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:10.762+0000 7f8635acb640 1 -- 172.21.15.146:0/1449041707 shutdown_connections 2024-09-11T07:59:10.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:10.762+0000 7f8635acb640 1 -- 172.21.15.146:0/1449041707 >> 172.21.15.146:0/1449041707 conn(0x7f86300f6c20 msgr2=0x7f8630106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:59:10.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:10.763+0000 7f8635acb640 1 -- 172.21.15.146:0/1449041707 shutdown_connections 2024-09-11T07:59:10.765 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:10.763+0000 7f8635acb640 1 -- 172.21.15.146:0/1449041707 wait complete. 2024-09-11T07:59:10.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:10 smithi102 ceph-mon[29849]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:10.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:10 smithi102 ceph-mon[41371]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:11.777 DEBUG:teuthology.orchestra.run.smithi146:osd.4> sudo journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.4.service 2024-09-11T07:59:11.779 INFO:tasks.cephadm:Deploying osd.5 on smithi146 with /dev/vg_nvme/lv_3... 2024-09-11T07:59:11.779 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-11T07:59:12.068 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:59:12.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:11 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:12.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:11 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:12.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:11 smithi102 ceph-mon[29849]: from='osd.4 v1:172.21.15.146:6800/2908355826' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:59:12.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:11 smithi102 ceph-mon[29849]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:59:12.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:11 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:12.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:11 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:12.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:11 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:12.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:11 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:12.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:11 smithi102 ceph-mon[41371]: from='osd.4 v1:172.21.15.146:6800/2908355826' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:59:12.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:11 smithi102 ceph-mon[41371]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:59:12.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:11 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:12.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:11 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:12.161 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:11 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:12.161 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:11 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:12.162 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:11 smithi146 ceph-mon[33158]: from='osd.4 v1:172.21.15.146:6800/2908355826' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:59:12.162 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:11 smithi146 ceph-mon[33158]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:59:12.162 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:11 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:12.162 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:11 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:13.050 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:12 smithi146 ceph-mon[33158]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:13.050 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:12 smithi146 ceph-mon[33158]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T07:59:13.050 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:12 smithi146 ceph-mon[33158]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T07:59:13.050 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:12 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:59:13.050 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:12 smithi146 ceph-mon[33158]: from='osd.4 v1:172.21.15.146:6800/2908355826' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T07:59:13.051 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:12 smithi146 ceph-mon[33158]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T07:59:13.051 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 11 07:59:12 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-4[38466]: 2024-09-11T07:59:12.771+0000 7f881943e640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T07:59:13.051 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 11 07:59:12 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-4[38466]: 2024-09-11T07:59:12.775+0000 7f8812271640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:59:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:12 smithi102 ceph-mon[29849]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:12 smithi102 ceph-mon[29849]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T07:59:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:12 smithi102 ceph-mon[29849]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T07:59:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:12 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:59:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:12 smithi102 ceph-mon[29849]: from='osd.4 v1:172.21.15.146:6800/2908355826' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T07:59:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:12 smithi102 ceph-mon[29849]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T07:59:13.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:12 smithi102 ceph-mon[41371]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:13.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:12 smithi102 ceph-mon[41371]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T07:59:13.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:12 smithi102 ceph-mon[41371]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T07:59:13.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:12 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:59:13.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:12 smithi102 ceph-mon[41371]: from='osd.4 v1:172.21.15.146:6800/2908355826' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T07:59:13.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:12 smithi102 ceph-mon[41371]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T07:59:13.858 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-11T07:59:13.922 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:13 smithi146 ceph-mon[33158]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-11T07:59:13.923 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:13 smithi146 ceph-mon[33158]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T07:59:13.923 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:13 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:59:13.923 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:13 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:59:13.942 INFO:teuthology.orchestra.run.smithi146.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-11T07:59:13.957 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10+0 records in 2024-09-11T07:59:13.957 INFO:teuthology.orchestra.run.smithi146.stderr:10+0 records out 2024-09-11T07:59:13.957 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124863 s, 840 MB/s 2024-09-11T07:59:13.957 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping successful for: 2024-09-11T07:59:14.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:13 smithi102 ceph-mon[29849]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-11T07:59:14.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:13 smithi102 ceph-mon[29849]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T07:59:14.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:13 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:59:14.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:13 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:59:14.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:13 smithi102 ceph-mon[41371]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-11T07:59:14.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:13 smithi102 ceph-mon[41371]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T07:59:14.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:13 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:59:14.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:13 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:59:14.703 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi146:vg_nvme/lv_3 2024-09-11T07:59:15.063 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:14 smithi146 ceph-mon[33158]: purged_snaps scrub starts 2024-09-11T07:59:15.063 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:14 smithi146 ceph-mon[33158]: purged_snaps scrub ok 2024-09-11T07:59:15.063 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:14 smithi146 ceph-mon[33158]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:15.063 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:14 smithi146 ceph-mon[33158]: osd.4 v1:172.21.15.146:6800/2908355826 boot 2024-09-11T07:59:15.063 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:14 smithi146 ceph-mon[33158]: osdmap e30: 5 total, 5 up, 5 in 2024-09-11T07:59:15.063 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:14 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:59:15.063 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:14 smithi146 ceph-mon[33158]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T07:59:15.131 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:59:15.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:14 smithi102 ceph-mon[29849]: purged_snaps scrub starts 2024-09-11T07:59:15.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:14 smithi102 ceph-mon[29849]: purged_snaps scrub ok 2024-09-11T07:59:15.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:14 smithi102 ceph-mon[29849]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:15.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:14 smithi102 ceph-mon[29849]: osd.4 v1:172.21.15.146:6800/2908355826 boot 2024-09-11T07:59:15.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:14 smithi102 ceph-mon[29849]: osdmap e30: 5 total, 5 up, 5 in 2024-09-11T07:59:15.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:14 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:59:15.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:14 smithi102 ceph-mon[29849]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T07:59:15.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:14 smithi102 ceph-mon[41371]: purged_snaps scrub starts 2024-09-11T07:59:15.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:14 smithi102 ceph-mon[41371]: purged_snaps scrub ok 2024-09-11T07:59:15.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:14 smithi102 ceph-mon[41371]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:59:15.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:14 smithi102 ceph-mon[41371]: osd.4 v1:172.21.15.146:6800/2908355826 boot 2024-09-11T07:59:15.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:14 smithi102 ceph-mon[41371]: osdmap e30: 5 total, 5 up, 5 in 2024-09-11T07:59:15.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:14 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:59:15.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:14 smithi102 ceph-mon[41371]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T07:59:16.925 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:16 smithi146 ceph-mon[33158]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:59:16.926 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:16 smithi146 ceph-mon[33158]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T07:59:17.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:16 smithi102 ceph-mon[29849]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:59:17.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:16 smithi102 ceph-mon[29849]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T07:59:17.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:16 smithi102 ceph-mon[41371]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:59:17.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:16 smithi102 ceph-mon[41371]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T07:59:17.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.734+0000 7f240b3b4640 1 -- 172.21.15.146:0/3542456735 >> v1:172.21.15.146:6789/0 conn(0x7f2404104670 legacy=0x7f2404106b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:59:17.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.734+0000 7f240b3b4640 1 -- 172.21.15.146:0/3542456735 shutdown_connections 2024-09-11T07:59:17.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.734+0000 7f240b3b4640 1 -- 172.21.15.146:0/3542456735 >> 172.21.15.146:0/3542456735 conn(0x7f24040f7d10 msgr2=0x7f24040fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:59:17.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.734+0000 7f240b3b4640 1 -- 172.21.15.146:0/3542456735 shutdown_connections 2024-09-11T07:59:17.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.734+0000 7f240b3b4640 1 -- 172.21.15.146:0/3542456735 wait complete. 2024-09-11T07:59:17.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.734+0000 7f240b3b4640 1 Processor -- start 2024-09-11T07:59:17.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.734+0000 7f240b3b4640 1 -- start start 2024-09-11T07:59:17.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.735+0000 7f240b3b4640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f240410d220 con 0x7f2404100a50 2024-09-11T07:59:17.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.735+0000 7f240b3b4640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2404118a20 con 0x7f24040fc550 2024-09-11T07:59:17.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.735+0000 7f240b3b4640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2404119bc0 con 0x7f2404104670 2024-09-11T07:59:17.739 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.735+0000 7f2409150640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f24040fc550 0x7f240410c690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:53374/0 (socket says 172.21.15.146:53374) 2024-09-11T07:59:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.735+0000 7f2409150640 1 -- 172.21.15.146:0/2934695889 learned_addr learned my addr 172.21.15.146:0/2934695889 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-11T07:59:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.735+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2054416878 0 0) 0x7f2404118a20 con 0x7f24040fc550 2024-09-11T07:59:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.735+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23f0003120 con 0x7f24040fc550 2024-09-11T07:59:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.735+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2729407626 0 0) 0x7f2404119bc0 con 0x7f2404104670 2024-09-11T07:59:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.735+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2404118a20 con 0x7f2404104670 2024-09-11T07:59:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.735+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616659251 0 0) 0x7f240410d220 con 0x7f2404100a50 2024-09-11T07:59:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.735+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2404119bc0 con 0x7f2404100a50 2024-09-11T07:59:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.735+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 783981362 0 0) 0x7f23f0003120 con 0x7f24040fc550 2024-09-11T07:59:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.735+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f240410d220 con 0x7f24040fc550 2024-09-11T07:59:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.735+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f24000018b0 con 0x7f24040fc550 2024-09-11T07:59:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.735+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1165629607 0 0) 0x7f2404118a20 con 0x7f2404104670 2024-09-11T07:59:17.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.735+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f23f0003120 con 0x7f2404104670 2024-09-11T07:59:17.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.735+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f23fc002f80 con 0x7f2404104670 2024-09-11T07:59:17.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.735+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3056439226 0 0) 0x7f2404119bc0 con 0x7f2404100a50 2024-09-11T07:59:17.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.735+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2404118a20 con 0x7f2404100a50 2024-09-11T07:59:17.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.735+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f23f40018c0 con 0x7f2404100a50 2024-09-11T07:59:17.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.735+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 <== mon.1 v1:172.21.15.146:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3392332814 0 0) 0x7f240410d220 con 0x7f24040fc550 2024-09-11T07:59:17.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.735+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 >> v1:172.21.15.102:6790/0 conn(0x7f2404104670 legacy=0x7f24041162f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:59:17.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.736+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 >> v1:172.21.15.102:6789/0 conn(0x7f2404100a50 legacy=0x7f2404112bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:59:17.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.736+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240411ada0 con 0x7f24040fc550 2024-09-11T07:59:17.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.736+0000 7f240b3b4640 1 -- 172.21.15.146:0/2934695889 --> v1:172.21.15.146:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2404119db0 con 0x7f24040fc550 2024-09-11T07:59:17.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.736+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 <== mon.1 v1:172.21.15.146:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f2400002270 con 0x7f24040fc550 2024-09-11T07:59:17.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.736+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 <== mon.1 v1:172.21.15.146:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f24000059a0 con 0x7f24040fc550 2024-09-11T07:59:17.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.736+0000 7f240b3b4640 1 -- 172.21.15.146:0/2934695889 --> v1:172.21.15.146:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f240411a200 con 0x7f24040fc550 2024-09-11T07:59:17.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.737+0000 7f240b3b4640 1 -- 172.21.15.146:0/2934695889 --> v1:172.21.15.146:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f240404bf60 con 0x7f24040fc550 2024-09-11T07:59:17.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.737+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 <== mon.1 v1:172.21.15.146:6789/0 7 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 1029906827 0 0) 0x7f2400001e20 con 0x7f24040fc550 2024-09-11T07:59:17.742 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.738+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 <== mon.1 v1:172.21.15.146:6789/0 8 ==== osd_map(32..32 src has 1..32) v4 ==== 3201+0+0 (unknown 643074102 0 0) 0x7f24000891f0 con 0x7f24040fc550 2024-09-11T07:59:17.743 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.742+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 <== mon.1 v1:172.21.15.146:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f2400056ad0 con 0x7f24040fc550 2024-09-11T07:59:17.907 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:17.906+0000 7f240b3b4640 1 -- 172.21.15.146:0/2934695889 --> v1:172.21.15.102:6800/2950771867 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f24040ffc00 con 0x7f23f006fd70 2024-09-11T07:59:18.610 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:18 smithi146 ceph-mon[33158]: Detected new or changed devices on smithi146 2024-09-11T07:59:18.610 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:18.610 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:18.610 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:59:18.610 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:18 smithi146 ceph-mon[33158]: Adjusting osd_memory_target on smithi146 to 17088M 2024-09-11T07:59:18.610 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:18.611 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:18.611 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:59:18.611 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:18.611 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:18.611 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:18.611 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:59:18.611 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:18.611 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:18 smithi146 ceph-mon[33158]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-09-11T07:59:18.611 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:18 smithi146 ceph-mon[33158]: from='client.24259 v1:172.21.15.146:0/2934695889' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:18.611 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:59:18.611 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:59:18.611 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:18.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[29849]: Detected new or changed devices on smithi146 2024-09-11T07:59:18.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:18.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:18.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:59:18.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[29849]: Adjusting osd_memory_target on smithi146 to 17088M 2024-09-11T07:59:18.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:18.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:18.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:59:18.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:18.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:18.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:18.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:59:18.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:18.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[29849]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-09-11T07:59:18.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[29849]: from='client.24259 v1:172.21.15.146:0/2934695889' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:18.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:59:18.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:59:18.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:18.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[41371]: Detected new or changed devices on smithi146 2024-09-11T07:59:18.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:18.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:18.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:59:18.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[41371]: Adjusting osd_memory_target on smithi146 to 17088M 2024-09-11T07:59:18.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:18.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:18.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:59:18.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:18.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:18.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:18.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:59:18.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:18.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[41371]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-09-11T07:59:18.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[41371]: from='client.24259 v1:172.21.15.146:0/2934695889' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:18.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:59:18.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:59:18.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:20.890 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:20 smithi146 ceph-mon[33158]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T07:59:20.891 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.146:0/4054198648' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6272822a-2ca3-45c8-a3c6-c00e26753989"}]: dispatch 2024-09-11T07:59:20.891 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:20 smithi146 ceph-mon[33158]: from='client.24265 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6272822a-2ca3-45c8-a3c6-c00e26753989"}]: dispatch 2024-09-11T07:59:20.891 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:20 smithi146 ceph-mon[33158]: from='client.24265 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6272822a-2ca3-45c8-a3c6-c00e26753989"}]': finished 2024-09-11T07:59:20.891 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:20 smithi146 ceph-mon[33158]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T07:59:20.891 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:20 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:59:20.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:20 smithi102 ceph-mon[29849]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T07:59:20.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.146:0/4054198648' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6272822a-2ca3-45c8-a3c6-c00e26753989"}]: dispatch 2024-09-11T07:59:20.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:20 smithi102 ceph-mon[29849]: from='client.24265 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6272822a-2ca3-45c8-a3c6-c00e26753989"}]: dispatch 2024-09-11T07:59:20.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:20 smithi102 ceph-mon[29849]: from='client.24265 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6272822a-2ca3-45c8-a3c6-c00e26753989"}]': finished 2024-09-11T07:59:20.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:20 smithi102 ceph-mon[29849]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T07:59:20.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:20 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:59:20.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:20 smithi102 ceph-mon[41371]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T07:59:20.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.146:0/4054198648' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6272822a-2ca3-45c8-a3c6-c00e26753989"}]: dispatch 2024-09-11T07:59:20.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:20 smithi102 ceph-mon[41371]: from='client.24265 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6272822a-2ca3-45c8-a3c6-c00e26753989"}]: dispatch 2024-09-11T07:59:20.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:20 smithi102 ceph-mon[41371]: from='client.24265 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6272822a-2ca3-45c8-a3c6-c00e26753989"}]': finished 2024-09-11T07:59:20.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:20 smithi102 ceph-mon[41371]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T07:59:20.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:20 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:59:21.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:21 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.146:0/2561846662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:59:21.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:21 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.146:0/2561846662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:59:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:21 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.146:0/2561846662' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:59:22.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:22 smithi102 ceph-mon[29849]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-11T07:59:22.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:22 smithi102 ceph-mon[41371]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-11T07:59:22.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:22 smithi146 ceph-mon[33158]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-11T07:59:24.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:24 smithi146 ceph-mon[33158]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T07:59:24.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:24 smithi102 ceph-mon[41371]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T07:59:24.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:24 smithi102 ceph-mon[29849]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T07:59:26.791 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:26 smithi146 ceph-mon[33158]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-11T07:59:26.791 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T07:59:26.791 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:26.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:26 smithi102 ceph-mon[41371]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-11T07:59:26.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T07:59:26.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:26.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:26 smithi102 ceph-mon[29849]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-11T07:59:26.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T07:59:26.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:27.806 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:27 smithi146 ceph-mon[33158]: Deploying daemon osd.5 on smithi146 2024-09-11T07:59:27.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:27 smithi102 ceph-mon[41371]: Deploying daemon osd.5 on smithi146 2024-09-11T07:59:27.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:27 smithi102 ceph-mon[29849]: Deploying daemon osd.5 on smithi146 2024-09-11T07:59:28.756 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:28 smithi146 ceph-mon[33158]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:59:28.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:28 smithi102 ceph-mon[41371]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:59:28.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:28 smithi102 ceph-mon[29849]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:59:30.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:30 smithi146 ceph-mon[33158]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:59:30.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:30 smithi102 ceph-mon[41371]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:59:30.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:30 smithi102 ceph-mon[29849]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:59:31.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:31 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:31.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:31 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:31.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:31 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:31.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:31 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:31.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:31 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:31.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:31 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:31.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:31 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:31.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:31 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:31.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:31 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:32.622 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:32 smithi146 ceph-mon[33158]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:59:32.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:32 smithi102 ceph-mon[29849]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:59:32.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:32 smithi102 ceph-mon[41371]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:59:34.006 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:34.006+0000 7f23f9ffb640 1 -- 172.21.15.146:0/2934695889 <== mgr.14150 v1:172.21.15.102:6800/2950771867 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 2814209452) 0x7f24040ffc00 con 0x7f23f006fd70 2024-09-11T07:59:34.007 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 5 on host 'smithi146' 2024-09-11T07:59:34.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:34.008+0000 7f240b3b4640 1 -- 172.21.15.146:0/2934695889 >> v1:172.21.15.102:6800/2950771867 conn(0x7f23f006fd70 legacy=0x7f23f0072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:59:34.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:34.008+0000 7f240b3b4640 1 -- 172.21.15.146:0/2934695889 >> v1:172.21.15.146:6789/0 conn(0x7f24040fc550 legacy=0x7f240410c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:59:34.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:34.008+0000 7f240b3b4640 1 -- 172.21.15.146:0/2934695889 shutdown_connections 2024-09-11T07:59:34.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:34.008+0000 7f240b3b4640 1 -- 172.21.15.146:0/2934695889 >> 172.21.15.146:0/2934695889 conn(0x7f24040f7d10 msgr2=0x7f2404106ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:59:34.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:34.008+0000 7f240b3b4640 1 -- 172.21.15.146:0/2934695889 shutdown_connections 2024-09-11T07:59:34.009 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:34.008+0000 7f240b3b4640 1 -- 172.21.15.146:0/2934695889 wait complete. 2024-09-11T07:59:34.621 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:34 smithi146 ceph-mon[33158]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:59:34.622 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:34 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:34.622 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:34 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:34.622 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:34 smithi146 ceph-mon[33158]: from='osd.5 v1:172.21.15.146:6804/1497668584' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:59:34.622 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:34 smithi146 ceph-mon[33158]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:59:34.645 DEBUG:teuthology.orchestra.run.smithi146:osd.5> sudo journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.5.service 2024-09-11T07:59:34.648 INFO:tasks.cephadm:Deploying osd.6 on smithi146 with /dev/vg_nvme/lv_2... 2024-09-11T07:59:34.648 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-11T07:59:34.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:34 smithi102 ceph-mon[29849]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:59:34.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:34 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:34.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:34 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:34.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:34 smithi102 ceph-mon[29849]: from='osd.5 v1:172.21.15.146:6804/1497668584' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:59:34.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:34 smithi102 ceph-mon[29849]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:59:34.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:34 smithi102 ceph-mon[41371]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:59:34.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:34 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:34.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:34 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:34.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:34 smithi102 ceph-mon[41371]: from='osd.5 v1:172.21.15.146:6804/1497668584' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:59:34.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:34 smithi102 ceph-mon[41371]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:59:34.940 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:59:35.969 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:35 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:35.969 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:35 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:35.969 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:35 smithi146 ceph-mon[33158]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T07:59:35.970 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:35 smithi146 ceph-mon[33158]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T07:59:35.970 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:35 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:59:35.970 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:35 smithi146 ceph-mon[33158]: from='osd.5 v1:172.21.15.146:6804/1497668584' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T07:59:35.970 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:35 smithi146 ceph-mon[33158]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T07:59:36.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:35 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:36.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:35 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:36.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:35 smithi102 ceph-mon[29849]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T07:59:36.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:35 smithi102 ceph-mon[29849]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T07:59:36.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:35 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:59:36.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:35 smithi102 ceph-mon[29849]: from='osd.5 v1:172.21.15.146:6804/1497668584' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T07:59:36.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:35 smithi102 ceph-mon[29849]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T07:59:36.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:35 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:36.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:35 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:36.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:35 smithi102 ceph-mon[41371]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T07:59:36.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:35 smithi102 ceph-mon[41371]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T07:59:36.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:35 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:59:36.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:35 smithi102 ceph-mon[41371]: from='osd.5 v1:172.21.15.146:6804/1497668584' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T07:59:36.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:35 smithi102 ceph-mon[41371]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T07:59:36.231 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 07:59:36 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T07:59:36.016+0000 7fc6a159a640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T07:59:36.231 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 07:59:36 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T07:59:36.023+0000 7fc69a3cd640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:59:36.825 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:36 smithi146 ceph-mon[33158]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:59:36.825 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:36 smithi146 ceph-mon[33158]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-11T07:59:36.825 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:36 smithi146 ceph-mon[33158]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T07:59:36.825 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:36 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:59:36.825 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:36 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:59:37.135 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-11T07:59:37.135 INFO:teuthology.orchestra.run.smithi146.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-11T07:59:37.135 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10+0 records in 2024-09-11T07:59:37.135 INFO:teuthology.orchestra.run.smithi146.stderr:10+0 records out 2024-09-11T07:59:37.135 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.014008 s, 749 MB/s 2024-09-11T07:59:37.135 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping successful for: 2024-09-11T07:59:37.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:36 smithi102 ceph-mon[29849]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:59:37.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:36 smithi102 ceph-mon[29849]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-11T07:59:37.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:36 smithi102 ceph-mon[29849]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T07:59:37.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:36 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:59:37.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:36 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:59:37.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:36 smithi102 ceph-mon[41371]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:59:37.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:36 smithi102 ceph-mon[41371]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-11T07:59:37.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:36 smithi102 ceph-mon[41371]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T07:59:37.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:36 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:59:37.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:36 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:59:37.720 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi146:vg_nvme/lv_2 2024-09-11T07:59:38.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:37 smithi102 ceph-mon[29849]: purged_snaps scrub starts 2024-09-11T07:59:38.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:37 smithi102 ceph-mon[29849]: purged_snaps scrub ok 2024-09-11T07:59:38.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:37 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:59:38.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:37 smithi102 ceph-mon[29849]: osd.5 v1:172.21.15.146:6804/1497668584 boot 2024-09-11T07:59:38.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:37 smithi102 ceph-mon[29849]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T07:59:38.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:37 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:59:38.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:37 smithi102 ceph-mon[41371]: purged_snaps scrub starts 2024-09-11T07:59:38.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:37 smithi102 ceph-mon[41371]: purged_snaps scrub ok 2024-09-11T07:59:38.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:37 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:59:38.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:37 smithi102 ceph-mon[41371]: osd.5 v1:172.21.15.146:6804/1497668584 boot 2024-09-11T07:59:38.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:37 smithi102 ceph-mon[41371]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T07:59:38.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:37 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:59:38.165 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:37 smithi146 ceph-mon[33158]: purged_snaps scrub starts 2024-09-11T07:59:38.165 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:37 smithi146 ceph-mon[33158]: purged_snaps scrub ok 2024-09-11T07:59:38.165 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:37 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:59:38.165 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:37 smithi146 ceph-mon[33158]: osd.5 v1:172.21.15.146:6804/1497668584 boot 2024-09-11T07:59:38.165 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:37 smithi146 ceph-mon[33158]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T07:59:38.166 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:37 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:59:38.726 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:59:38.974 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:38 smithi146 ceph-mon[33158]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:59:38.974 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:38 smithi146 ceph-mon[33158]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T07:59:39.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:38 smithi102 ceph-mon[29849]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:59:39.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:38 smithi102 ceph-mon[29849]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T07:59:39.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:38 smithi102 ceph-mon[41371]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:59:39.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:38 smithi102 ceph-mon[41371]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T07:59:40.129 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:40 smithi146 ceph-mon[33158]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T07:59:40.129 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:40 smithi146 ceph-mon[33158]: pgmap v126: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:59:40.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:40 smithi102 ceph-mon[41371]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T07:59:40.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:40 smithi102 ceph-mon[41371]: pgmap v126: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:59:40.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:40 smithi102 ceph-mon[29849]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T07:59:40.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:40 smithi102 ceph-mon[29849]: pgmap v126: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:59:40.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.827+0000 7f15f2805640 1 -- 172.21.15.146:0/2125329751 >> v1:172.21.15.146:6789/0 conn(0x7f15ec103580 legacy=0x7f15ec105a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:59:40.839 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.827+0000 7f15f2805640 1 -- 172.21.15.146:0/2125329751 shutdown_connections 2024-09-11T07:59:40.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.827+0000 7f15f2805640 1 -- 172.21.15.146:0/2125329751 >> 172.21.15.146:0/2125329751 conn(0x7f15ec0f6c20 msgr2=0x7f15ec0f9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:59:40.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.827+0000 7f15f2805640 1 -- 172.21.15.146:0/2125329751 shutdown_connections 2024-09-11T07:59:40.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.827+0000 7f15f2805640 1 -- 172.21.15.146:0/2125329751 wait complete. 2024-09-11T07:59:40.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.827+0000 7f15f2805640 1 Processor -- start 2024-09-11T07:59:40.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.828+0000 7f15f2805640 1 -- start start 2024-09-11T07:59:40.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.828+0000 7f15f2805640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f15ec10d090 con 0x7f15ec0fb460 2024-09-11T07:59:40.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.828+0000 7f15f2805640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f15ec118890 con 0x7f15ec0ff960 2024-09-11T07:59:40.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.828+0000 7f15f2805640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f15ec119a30 con 0x7f15ec103580 2024-09-11T07:59:40.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.828+0000 7f15eb7fe640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f15ec0ff960 0x7f15ec112a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:37190/0 (socket says 172.21.15.146:37190) 2024-09-11T07:59:40.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.828+0000 7f15eb7fe640 1 -- 172.21.15.146:0/2068856116 learned_addr learned my addr 172.21.15.146:0/2068856116 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-11T07:59:40.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.828+0000 7f15e8ff9640 1 -- 172.21.15.146:0/2068856116 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1986552173 0 0) 0x7f15ec118890 con 0x7f15ec0ff960 2024-09-11T07:59:40.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.828+0000 7f15e8ff9640 1 -- 172.21.15.146:0/2068856116 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15d4003120 con 0x7f15ec0ff960 2024-09-11T07:59:40.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.828+0000 7f15e8ff9640 1 -- 172.21.15.146:0/2068856116 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 843463527 0 0) 0x7f15ec119a30 con 0x7f15ec103580 2024-09-11T07:59:40.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.828+0000 7f15e8ff9640 1 -- 172.21.15.146:0/2068856116 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15ec118890 con 0x7f15ec103580 2024-09-11T07:59:40.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.828+0000 7f15e8ff9640 1 -- 172.21.15.146:0/2068856116 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3512152400 0 0) 0x7f15ec10d090 con 0x7f15ec0fb460 2024-09-11T07:59:40.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.828+0000 7f15e8ff9640 1 -- 172.21.15.146:0/2068856116 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15ec119a30 con 0x7f15ec0fb460 2024-09-11T07:59:40.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.828+0000 7f15e8ff9640 1 -- 172.21.15.146:0/2068856116 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4158130591 0 0) 0x7f15d4003120 con 0x7f15ec0ff960 2024-09-11T07:59:40.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.828+0000 7f15e8ff9640 1 -- 172.21.15.146:0/2068856116 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f15ec10d090 con 0x7f15ec0ff960 2024-09-11T07:59:40.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.828+0000 7f15e8ff9640 1 -- 172.21.15.146:0/2068856116 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f15e0001910 con 0x7f15ec0ff960 2024-09-11T07:59:40.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.828+0000 7f15e8ff9640 1 -- 172.21.15.146:0/2068856116 <== mon.1 v1:172.21.15.146:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 975885177 0 0) 0x7f15ec10d090 con 0x7f15ec0ff960 2024-09-11T07:59:40.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.829+0000 7f15e8ff9640 1 -- 172.21.15.146:0/2068856116 >> v1:172.21.15.102:6790/0 conn(0x7f15ec103580 legacy=0x7f15ec116160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:59:40.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.829+0000 7f15e8ff9640 1 -- 172.21.15.146:0/2068856116 >> v1:172.21.15.102:6789/0 conn(0x7f15ec0fb460 legacy=0x7f15ec10c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:59:40.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.829+0000 7f15e8ff9640 1 -- 172.21.15.146:0/2068856116 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ec11ac10 con 0x7f15ec0ff960 2024-09-11T07:59:40.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.829+0000 7f15f2805640 1 -- 172.21.15.146:0/2068856116 --> v1:172.21.15.146:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ec1178e0 con 0x7f15ec0ff960 2024-09-11T07:59:40.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.829+0000 7f15f2805640 1 -- 172.21.15.146:0/2068856116 --> v1:172.21.15.146:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ec117db0 con 0x7f15ec0ff960 2024-09-11T07:59:40.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.829+0000 7f15e8ff9640 1 -- 172.21.15.146:0/2068856116 <== mon.1 v1:172.21.15.146:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f15e00023d0 con 0x7f15ec0ff960 2024-09-11T07:59:40.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.829+0000 7f15e8ff9640 1 -- 172.21.15.146:0/2068856116 <== mon.1 v1:172.21.15.146:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f15e0005a10 con 0x7f15ec0ff960 2024-09-11T07:59:40.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.830+0000 7f15f2805640 1 -- 172.21.15.146:0/2068856116 --> v1:172.21.15.146:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d8005040 con 0x7f15ec0ff960 2024-09-11T07:59:40.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.830+0000 7f15e8ff9640 1 -- 172.21.15.146:0/2068856116 <== mon.1 v1:172.21.15.146:6789/0 7 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 1029906827 0 0) 0x7f15e001be00 con 0x7f15ec0ff960 2024-09-11T07:59:40.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.831+0000 7f15e8ff9640 1 -- 172.21.15.146:0/2068856116 <== mon.1 v1:172.21.15.146:6789/0 8 ==== osd_map(38..38 src has 1..38) v4 ==== 3493+0+0 (unknown 517913474 0 0) 0x7f15e00893d0 con 0x7f15ec0ff960 2024-09-11T07:59:40.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.834+0000 7f15e8ff9640 1 -- 172.21.15.146:0/2068856116 <== mon.1 v1:172.21.15.146:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f15e0056b30 con 0x7f15ec0ff960 2024-09-11T07:59:40.998 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:40.997+0000 7f15f2805640 1 -- 172.21.15.146:0/2068856116 --> v1:172.21.15.102:6800/2950771867 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f15d8000bc0 con 0x7f15d406f930 2024-09-11T07:59:41.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[29849]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T07:59:41.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[29849]: Detected new or changed devices on smithi146 2024-09-11T07:59:41.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:41.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:41.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:59:41.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:59:41.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[29849]: Adjusting osd_memory_target on smithi146 to 8544M 2024-09-11T07:59:41.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:41.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:41.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:59:41.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:41.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:41.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:41.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:59:41.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:41.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:59:41.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:59:41.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:41.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[41371]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T07:59:41.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[41371]: Detected new or changed devices on smithi146 2024-09-11T07:59:41.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:41.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:41.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:59:41.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:59:41.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[41371]: Adjusting osd_memory_target on smithi146 to 8544M 2024-09-11T07:59:41.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:41.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:41.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:59:41.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:41.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:41.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:41.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:59:41.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:41.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:59:41.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:59:41.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:41 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:41.406 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:41 smithi146 ceph-mon[33158]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T07:59:41.406 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:41 smithi146 ceph-mon[33158]: Detected new or changed devices on smithi146 2024-09-11T07:59:41.406 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:41 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:41.406 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:41 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:41.406 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:41 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:59:41.406 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:41 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:59:41.406 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:41 smithi146 ceph-mon[33158]: Adjusting osd_memory_target on smithi146 to 8544M 2024-09-11T07:59:41.407 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:41 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:41.407 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:41 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:41.407 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:41 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:59:41.407 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:41 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:41.407 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:41 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:41.407 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:41 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:41.407 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:41 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:59:41.407 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:41 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:41.407 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:41 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:59:41.407 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:41 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:59:41.407 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:41 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:42.387 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:42 smithi146 ceph-mon[33158]: from='client.24286 v1:172.21.15.146:0/2068856116' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:42.388 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:42 smithi146 ceph-mon[33158]: pgmap v127: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:59:42.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:42 smithi102 ceph-mon[41371]: from='client.24286 v1:172.21.15.146:0/2068856116' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:42.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:42 smithi102 ceph-mon[41371]: pgmap v127: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:59:42.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:42 smithi102 ceph-mon[29849]: from='client.24286 v1:172.21.15.146:0/2068856116' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:59:42.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:42 smithi102 ceph-mon[29849]: pgmap v127: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:59:43.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:43 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.146:0/4004690012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c3a857f-a8d5-46d0-8fd5-bcc759074f72"}]: dispatch 2024-09-11T07:59:43.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:43 smithi102 ceph-mon[41371]: from='client.24292 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c3a857f-a8d5-46d0-8fd5-bcc759074f72"}]: dispatch 2024-09-11T07:59:43.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:43 smithi102 ceph-mon[41371]: from='client.24292 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c3a857f-a8d5-46d0-8fd5-bcc759074f72"}]': finished 2024-09-11T07:59:43.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:43 smithi102 ceph-mon[41371]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T07:59:43.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:43 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:59:43.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:43 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.146:0/4004690012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c3a857f-a8d5-46d0-8fd5-bcc759074f72"}]: dispatch 2024-09-11T07:59:43.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:43 smithi102 ceph-mon[29849]: from='client.24292 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c3a857f-a8d5-46d0-8fd5-bcc759074f72"}]: dispatch 2024-09-11T07:59:43.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:43 smithi102 ceph-mon[29849]: from='client.24292 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c3a857f-a8d5-46d0-8fd5-bcc759074f72"}]': finished 2024-09-11T07:59:43.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:43 smithi102 ceph-mon[29849]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T07:59:43.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:43 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:59:43.658 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:43 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.146:0/4004690012' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c3a857f-a8d5-46d0-8fd5-bcc759074f72"}]: dispatch 2024-09-11T07:59:43.659 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:43 smithi146 ceph-mon[33158]: from='client.24292 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c3a857f-a8d5-46d0-8fd5-bcc759074f72"}]: dispatch 2024-09-11T07:59:43.659 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:43 smithi146 ceph-mon[33158]: from='client.24292 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c3a857f-a8d5-46d0-8fd5-bcc759074f72"}]': finished 2024-09-11T07:59:43.659 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:43 smithi146 ceph-mon[33158]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T07:59:43.659 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:43 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:59:44.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:44 smithi102 ceph-mon[29849]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:59:44.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:44 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.146:0/4078749856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:59:44.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:44 smithi102 ceph-mon[41371]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:59:44.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:44 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.146:0/4078749856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:59:44.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:44 smithi146 ceph-mon[33158]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:59:44.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:44 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.146:0/4078749856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:59:45.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:45 smithi102 ceph-mon[29849]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T07:59:45.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:45 smithi102 ceph-mon[29849]: Cluster is now healthy 2024-09-11T07:59:45.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:45 smithi102 ceph-mon[41371]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T07:59:45.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:45 smithi102 ceph-mon[41371]: Cluster is now healthy 2024-09-11T07:59:45.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:45 smithi146 ceph-mon[33158]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T07:59:45.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:45 smithi146 ceph-mon[33158]: Cluster is now healthy 2024-09-11T07:59:46.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:46 smithi102 ceph-mon[29849]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-11T07:59:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:46 smithi102 ceph-mon[41371]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-11T07:59:46.670 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:46 smithi146 ceph-mon[33158]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-11T07:59:48.662 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:48 smithi146 ceph-mon[33158]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-11T07:59:48.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:48 smithi102 ceph-mon[41371]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-11T07:59:48.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:48 smithi102 ceph-mon[29849]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-11T07:59:49.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:49 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T07:59:49.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:49 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:49.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:49 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T07:59:49.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:49 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:49.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:49 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T07:59:49.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:49 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:50.681 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:50 smithi146 ceph-mon[33158]: Deploying daemon osd.6 on smithi146 2024-09-11T07:59:50.681 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:50 smithi146 ceph-mon[33158]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:59:50.797 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:50 smithi102 ceph-mon[29849]: Deploying daemon osd.6 on smithi146 2024-09-11T07:59:50.797 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:50 smithi102 ceph-mon[29849]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:59:50.797 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:50 smithi102 ceph-mon[41371]: Deploying daemon osd.6 on smithi146 2024-09-11T07:59:50.797 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:50 smithi102 ceph-mon[41371]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:59:52.755 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:52 smithi146 ceph-mon[33158]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:59:52.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:52 smithi102 ceph-mon[29849]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:59:52.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:52 smithi102 ceph-mon[41371]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:59:54.809 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:54 smithi146 ceph-mon[33158]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:59:54.809 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:54 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:54.809 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:54 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:54.809 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:54 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:54.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:54 smithi102 ceph-mon[41371]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:59:54.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:54 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:54.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:54 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:54.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:54 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:54.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:54 smithi102 ceph-mon[29849]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:59:54.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:54 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:54.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:54 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:54.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:54 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:56.610 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:56 smithi146 ceph-mon[33158]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:59:56.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:56 smithi102 ceph-mon[29849]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:59:56.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:56 smithi102 ceph-mon[41371]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:59:57.549 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:57.547+0000 7f15e8ff9640 1 -- 172.21.15.146:0/2068856116 <== mgr.14150 v1:172.21.15.102:6800/2950771867 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 1132682864) 0x7f15d8000bc0 con 0x7f15d406f930 2024-09-11T07:59:57.549 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 6 on host 'smithi146' 2024-09-11T07:59:57.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:57.549+0000 7f15f2805640 1 -- 172.21.15.146:0/2068856116 >> v1:172.21.15.102:6800/2950771867 conn(0x7f15d406f930 legacy=0x7f15d4071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:59:57.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:57.549+0000 7f15f2805640 1 -- 172.21.15.146:0/2068856116 >> v1:172.21.15.146:6789/0 conn(0x7f15ec0ff960 legacy=0x7f15ec112a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T07:59:57.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:57.549+0000 7f15f2805640 1 -- 172.21.15.146:0/2068856116 shutdown_connections 2024-09-11T07:59:57.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:57.549+0000 7f15f2805640 1 -- 172.21.15.146:0/2068856116 >> 172.21.15.146:0/2068856116 conn(0x7f15ec0f6c20 msgr2=0x7f15ec0660e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T07:59:57.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:57.549+0000 7f15f2805640 1 -- 172.21.15.146:0/2068856116 shutdown_connections 2024-09-11T07:59:57.551 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T07:59:57.549+0000 7f15f2805640 1 -- 172.21.15.146:0/2068856116 wait complete. 2024-09-11T07:59:58.143 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:57 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:58.143 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:57 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:58.144 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:57 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:58.144 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:57 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:59:58.144 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:57 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:58.144 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:57 smithi146 ceph-mon[33158]: from='osd.6 v1:172.21.15.146:6808/85686736' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:59:58.144 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:57 smithi146 ceph-mon[33158]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:59:58.144 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:57 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:58.144 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:57 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:58.144 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:57 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:58.168 DEBUG:teuthology.orchestra.run.smithi146:osd.6> sudo journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.6.service 2024-09-11T07:59:58.171 INFO:tasks.cephadm:Deploying osd.7 on smithi146 with /dev/vg_nvme/lv_1... 2024-09-11T07:59:58.171 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-11T07:59:58.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:57 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:58.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:57 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:58.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:57 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:58.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:57 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:59:58.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:57 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:58.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:57 smithi102 ceph-mon[29849]: from='osd.6 v1:172.21.15.146:6808/85686736' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:59:58.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:57 smithi102 ceph-mon[29849]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:59:58.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:57 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:58.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:57 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:58.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:57 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:58.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:57 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:58.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:57 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:58.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:57 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:59:58.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:57 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:59:58.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:57 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:58.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:57 smithi102 ceph-mon[41371]: from='osd.6 v1:172.21.15.146:6808/85686736' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:59:58.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:57 smithi102 ceph-mon[41371]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:59:58.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:57 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:59:58.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:57 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:58.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:57 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T07:59:58.972 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:59:59.209 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:58 smithi146 ceph-mon[33158]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:59:59.210 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:58 smithi146 ceph-mon[33158]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T07:59:59.210 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:58 smithi146 ceph-mon[33158]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T07:59:59.210 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:58 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:59:59.210 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:58 smithi146 ceph-mon[33158]: from='osd.6 v1:172.21.15.146:6808/85686736' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T07:59:59.210 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:58 smithi146 ceph-mon[33158]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T07:59:59.210 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 07:59:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T07:59:58.939+0000 7fcd189be640 -1 osd.6 0 waiting for initial osdmap 2024-09-11T07:59:59.210 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 07:59:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T07:59:58.945+0000 7fcd127d1640 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:59:59.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:58 smithi102 ceph-mon[41371]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:59:59.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:58 smithi102 ceph-mon[41371]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T07:59:59.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:58 smithi102 ceph-mon[41371]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T07:59:59.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:58 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:59:59.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:58 smithi102 ceph-mon[41371]: from='osd.6 v1:172.21.15.146:6808/85686736' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T07:59:59.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:58 smithi102 ceph-mon[41371]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T07:59:59.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:58 smithi102 ceph-mon[29849]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:59:59.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:58 smithi102 ceph-mon[29849]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T07:59:59.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:58 smithi102 ceph-mon[29849]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T07:59:59.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:58 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:59:59.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:58 smithi102 ceph-mon[29849]: from='osd.6 v1:172.21.15.146:6808/85686736' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T07:59:59.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:58 smithi102 ceph-mon[29849]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T08:00:00.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:59 smithi146 ceph-mon[33158]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-11T08:00:00.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:59 smithi146 ceph-mon[33158]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T08:00:00.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:59 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:00:00.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 07:59:59 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:00:00.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:59 smithi102 ceph-mon[29849]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-11T08:00:00.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:59 smithi102 ceph-mon[29849]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T08:00:00.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:59 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:00:00.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 07:59:59 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:00:00.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:59 smithi102 ceph-mon[41371]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-11T08:00:00.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:59 smithi102 ceph-mon[41371]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T08:00:00.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:59 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:00:00.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 07:59:59 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:00:01.087 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:00 smithi146 ceph-mon[33158]: purged_snaps scrub starts 2024-09-11T08:00:01.087 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:00 smithi146 ceph-mon[33158]: purged_snaps scrub ok 2024-09-11T08:00:01.087 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:00 smithi146 ceph-mon[33158]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:00:01.087 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:00 smithi146 ceph-mon[33158]: osd.6 v1:172.21.15.146:6808/85686736 boot 2024-09-11T08:00:01.087 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:00 smithi146 ceph-mon[33158]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T08:00:01.087 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:00 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:00:01.087 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:00 smithi146 ceph-mon[33158]: overall HEALTH_OK 2024-09-11T08:00:01.087 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:00 smithi146 ceph-mon[33158]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T08:00:01.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:00 smithi102 ceph-mon[29849]: purged_snaps scrub starts 2024-09-11T08:00:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:00 smithi102 ceph-mon[29849]: purged_snaps scrub ok 2024-09-11T08:00:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:00 smithi102 ceph-mon[29849]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:00:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:00 smithi102 ceph-mon[29849]: osd.6 v1:172.21.15.146:6808/85686736 boot 2024-09-11T08:00:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:00 smithi102 ceph-mon[29849]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T08:00:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:00 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:00:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:00 smithi102 ceph-mon[29849]: overall HEALTH_OK 2024-09-11T08:00:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:00 smithi102 ceph-mon[29849]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T08:00:01.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:00 smithi102 ceph-mon[41371]: purged_snaps scrub starts 2024-09-11T08:00:01.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:00 smithi102 ceph-mon[41371]: purged_snaps scrub ok 2024-09-11T08:00:01.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:00 smithi102 ceph-mon[41371]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T08:00:01.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:00 smithi102 ceph-mon[41371]: osd.6 v1:172.21.15.146:6808/85686736 boot 2024-09-11T08:00:01.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:00 smithi102 ceph-mon[41371]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T08:00:01.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:00 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:00:01.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:00 smithi102 ceph-mon[41371]: overall HEALTH_OK 2024-09-11T08:00:01.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:00 smithi102 ceph-mon[41371]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T08:00:02.164 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-11T08:00:02.252 INFO:teuthology.orchestra.run.smithi146.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-11T08:00:02.265 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10+0 records in 2024-09-11T08:00:02.266 INFO:teuthology.orchestra.run.smithi146.stderr:10+0 records out 2024-09-11T08:00:02.266 INFO:teuthology.orchestra.run.smithi146.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114316 s, 917 MB/s 2024-09-11T08:00:02.266 INFO:teuthology.orchestra.run.smithi146.stderr:--> Zapping successful for: 2024-09-11T08:00:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[29849]: Detected new or changed devices on smithi146 2024-09-11T08:00:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:00:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:00:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:00:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[29849]: Adjusting osd_memory_target on smithi146 to 5696M 2024-09-11T08:00:02.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:02.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:02.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:00:02.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:02.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[29849]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:02.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[29849]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T08:00:02.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[41371]: Detected new or changed devices on smithi146 2024-09-11T08:00:02.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:02.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:02.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:00:02.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:00:02.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:00:02.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[41371]: Adjusting osd_memory_target on smithi146 to 5696M 2024-09-11T08:00:02.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:02.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:02.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:00:02.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:02.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[41371]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:02.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:02 smithi102 ceph-mon[41371]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T08:00:02.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:02 smithi146 ceph-mon[33158]: Detected new or changed devices on smithi146 2024-09-11T08:00:02.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:02 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:02.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:02 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:02.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:02 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:00:02.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:02 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:00:02.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:02 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:00:02.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:02 smithi146 ceph-mon[33158]: Adjusting osd_memory_target on smithi146 to 5696M 2024-09-11T08:00:02.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:02 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:02.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:02 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:02.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:02 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:00:02.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:02 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:02.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:02 smithi146 ceph-mon[33158]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:02.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:02 smithi146 ceph-mon[33158]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T08:00:02.928 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi146:vg_nvme/lv_1 2024-09-11T08:00:03.171 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:00:04.746 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:04 smithi146 ceph-mon[33158]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:04.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.735+0000 7f673ab0d640 1 -- 172.21.15.146:0/41612524 >> v1:172.21.15.146:6789/0 conn(0x7f6734100a10 legacy=0x7f6734102df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:04.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.736+0000 7f673ab0d640 1 -- 172.21.15.146:0/41612524 shutdown_connections 2024-09-11T08:00:04.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.736+0000 7f673ab0d640 1 -- 172.21.15.146:0/41612524 >> 172.21.15.146:0/41612524 conn(0x7f67340f7cd0 msgr2=0x7f67340fa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:00:04.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.736+0000 7f673ab0d640 1 -- 172.21.15.146:0/41612524 shutdown_connections 2024-09-11T08:00:04.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.736+0000 7f673ab0d640 1 -- 172.21.15.146:0/41612524 wait complete. 2024-09-11T08:00:04.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.736+0000 7f673ab0d640 1 Processor -- start 2024-09-11T08:00:04.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.736+0000 7f673ab0d640 1 -- start start 2024-09-11T08:00:04.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.736+0000 7f673ab0d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6734119710 con 0x7f6734100a10 2024-09-11T08:00:04.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.736+0000 7f673ab0d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f673411a910 con 0x7f67340fc510 2024-09-11T08:00:04.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.736+0000 7f673ab0d640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f673411bb10 con 0x7f6734104630 2024-09-11T08:00:04.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.736+0000 7f67388a9640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f67340fc510 0x7f6734071320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:41894/0 (socket says 172.21.15.146:41894) 2024-09-11T08:00:04.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.736+0000 7f67388a9640 1 -- 172.21.15.146:0/151905749 learned_addr learned my addr 172.21.15.146:0/151905749 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-11T08:00:04.767 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.737+0000 7f67317fa640 1 -- 172.21.15.146:0/151905749 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3090969545 0 0) 0x7f673411a910 con 0x7f67340fc510 2024-09-11T08:00:04.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.737+0000 7f67317fa640 1 -- 172.21.15.146:0/151905749 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6724003120 con 0x7f67340fc510 2024-09-11T08:00:04.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.737+0000 7f67317fa640 1 -- 172.21.15.146:0/151905749 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2572858488 0 0) 0x7f6734119710 con 0x7f6734100a10 2024-09-11T08:00:04.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.737+0000 7f67317fa640 1 -- 172.21.15.146:0/151905749 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f673411a910 con 0x7f6734100a10 2024-09-11T08:00:04.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.737+0000 7f67317fa640 1 -- 172.21.15.146:0/151905749 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 268946132 0 0) 0x7f673411bb10 con 0x7f6734104630 2024-09-11T08:00:04.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.737+0000 7f67317fa640 1 -- 172.21.15.146:0/151905749 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6734119710 con 0x7f6734104630 2024-09-11T08:00:04.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.737+0000 7f67317fa640 1 -- 172.21.15.146:0/151905749 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 751949302 0 0) 0x7f6724003120 con 0x7f67340fc510 2024-09-11T08:00:04.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.737+0000 7f67317fa640 1 -- 172.21.15.146:0/151905749 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f673411bb10 con 0x7f67340fc510 2024-09-11T08:00:04.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.737+0000 7f67317fa640 1 -- 172.21.15.146:0/151905749 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f6728002900 con 0x7f67340fc510 2024-09-11T08:00:04.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.737+0000 7f67317fa640 1 -- 172.21.15.146:0/151905749 <== mon.1 v1:172.21.15.146:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2639620003 0 0) 0x7f673411bb10 con 0x7f67340fc510 2024-09-11T08:00:04.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.737+0000 7f67317fa640 1 -- 172.21.15.146:0/151905749 >> v1:172.21.15.102:6790/0 conn(0x7f6734104630 legacy=0x7f6734117e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:04.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.737+0000 7f67317fa640 1 -- 172.21.15.146:0/151905749 >> v1:172.21.15.102:6789/0 conn(0x7f6734100a10 legacy=0x7f6734114550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:04.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.737+0000 7f67317fa640 1 -- 172.21.15.146:0/151905749 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673411cd10 con 0x7f67340fc510 2024-09-11T08:00:04.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.737+0000 7f673ab0d640 1 -- 172.21.15.146:0/151905749 --> v1:172.21.15.146:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6734119900 con 0x7f67340fc510 2024-09-11T08:00:04.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.737+0000 7f673ab0d640 1 -- 172.21.15.146:0/151905749 --> v1:172.21.15.146:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6734119d50 con 0x7f67340fc510 2024-09-11T08:00:04.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.737+0000 7f67317fa640 1 -- 172.21.15.146:0/151905749 <== mon.1 v1:172.21.15.146:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f6728002c90 con 0x7f67340fc510 2024-09-11T08:00:04.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.737+0000 7f67317fa640 1 -- 172.21.15.146:0/151905749 <== mon.1 v1:172.21.15.146:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f6728005af0 con 0x7f67340fc510 2024-09-11T08:00:04.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.738+0000 7f673ab0d640 1 -- 172.21.15.146:0/151905749 --> v1:172.21.15.146:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6718005040 con 0x7f67340fc510 2024-09-11T08:00:04.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.739+0000 7f67317fa640 1 -- 172.21.15.146:0/151905749 <== mon.1 v1:172.21.15.146:6789/0 7 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 1029906827 0 0) 0x7f6728001240 con 0x7f67340fc510 2024-09-11T08:00:04.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.739+0000 7f67317fa640 1 -- 172.21.15.146:0/151905749 <== mon.1 v1:172.21.15.146:6789/0 8 ==== osd_map(44..44 src has 1..44) v4 ==== 3785+0+0 (unknown 2988566866 0 0) 0x7f6728089470 con 0x7f67340fc510 2024-09-11T08:00:04.769 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.743+0000 7f67317fa640 1 -- 172.21.15.146:0/151905749 <== mon.1 v1:172.21.15.146:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f6728056b00 con 0x7f67340fc510 2024-09-11T08:00:04.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:04 smithi102 ceph-mon[29849]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:04.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:04 smithi102 ceph-mon[41371]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:04.911 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:04.910+0000 7f673ab0d640 1 -- 172.21.15.146:0/151905749 --> v1:172.21.15.102:6800/2950771867 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6718000bc0 con 0x7f672406f760 2024-09-11T08:00:05.608 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:05 smithi146 ceph-mon[33158]: from='client.24313 v1:172.21.15.146:0/151905749' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:00:05.608 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:05 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:00:05.608 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:05 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:00:05.608 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:05 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:05.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:05 smithi102 ceph-mon[41371]: from='client.24313 v1:172.21.15.146:0/151905749' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:00:05.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:05 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:00:05.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:05 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:00:05.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:05 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:05.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:05 smithi102 ceph-mon[29849]: from='client.24313 v1:172.21.15.146:0/151905749' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:00:05.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:05 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T08:00:05.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:05 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T08:00:05.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:05 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:06.642 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:06 smithi146 ceph-mon[33158]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:06.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:06 smithi102 ceph-mon[29849]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:06.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:06 smithi102 ceph-mon[41371]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:07.835 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.146:0/3234899052' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca7e36e9-ca16-4d15-8264-deed759fdbf2"}]: dispatch 2024-09-11T08:00:07.836 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:07 smithi146 ceph-mon[33158]: from='client.24319 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca7e36e9-ca16-4d15-8264-deed759fdbf2"}]: dispatch 2024-09-11T08:00:07.836 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:07 smithi146 ceph-mon[33158]: from='client.24319 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca7e36e9-ca16-4d15-8264-deed759fdbf2"}]': finished 2024-09-11T08:00:07.836 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:07 smithi146 ceph-mon[33158]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T08:00:07.836 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:07 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:00:07.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.146:0/3234899052' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca7e36e9-ca16-4d15-8264-deed759fdbf2"}]: dispatch 2024-09-11T08:00:07.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:07 smithi102 ceph-mon[41371]: from='client.24319 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca7e36e9-ca16-4d15-8264-deed759fdbf2"}]: dispatch 2024-09-11T08:00:07.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:07 smithi102 ceph-mon[41371]: from='client.24319 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca7e36e9-ca16-4d15-8264-deed759fdbf2"}]': finished 2024-09-11T08:00:07.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:07 smithi102 ceph-mon[41371]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T08:00:07.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:07 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:00:07.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.146:0/3234899052' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca7e36e9-ca16-4d15-8264-deed759fdbf2"}]: dispatch 2024-09-11T08:00:07.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:07 smithi102 ceph-mon[29849]: from='client.24319 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ca7e36e9-ca16-4d15-8264-deed759fdbf2"}]: dispatch 2024-09-11T08:00:07.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:07 smithi102 ceph-mon[29849]: from='client.24319 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ca7e36e9-ca16-4d15-8264-deed759fdbf2"}]': finished 2024-09-11T08:00:07.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:07 smithi102 ceph-mon[29849]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T08:00:07.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:07 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:00:08.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:08 smithi102 ceph-mon[41371]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:08.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:08 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.146:0/489892700' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:00:08.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:08 smithi102 ceph-mon[29849]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:08.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:08 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.146:0/489892700' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:00:08.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:08 smithi146 ceph-mon[33158]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:08.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:08 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.146:0/489892700' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T08:00:10.845 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:10 smithi146 ceph-mon[33158]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:10.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:10 smithi102 ceph-mon[29849]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:10.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:10 smithi102 ceph-mon[41371]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:12.565 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:12 smithi146 ceph-mon[33158]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:12.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:12 smithi102 ceph-mon[29849]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:12.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:12 smithi102 ceph-mon[41371]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:14.890 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:14 smithi146 ceph-mon[33158]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:14.890 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:14 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T08:00:14.890 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:14 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:14.890 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:14 smithi146 ceph-mon[33158]: Deploying daemon osd.7 on smithi146 2024-09-11T08:00:14.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:14 smithi102 ceph-mon[29849]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:14.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:14 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T08:00:14.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:14 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:14.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:14 smithi102 ceph-mon[29849]: Deploying daemon osd.7 on smithi146 2024-09-11T08:00:14.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:14 smithi102 ceph-mon[41371]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:14.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:14 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T08:00:14.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:14 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:14.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:14 smithi102 ceph-mon[41371]: Deploying daemon osd.7 on smithi146 2024-09-11T08:00:16.787 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:16 smithi146 ceph-mon[33158]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:16.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:16 smithi102 ceph-mon[29849]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:16 smithi102 ceph-mon[41371]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:18.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:18 smithi146 ceph-mon[33158]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:18.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:00:18.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:18.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:18 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:18.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:18 smithi102 ceph-mon[29849]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:18.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:00:18.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:18.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:18 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:18.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:18 smithi102 ceph-mon[41371]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:18.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:00:18.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:18.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:18 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:20.820 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:20 smithi146 ceph-mon[33158]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:20.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:20 smithi102 ceph-mon[29849]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:20.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:20 smithi102 ceph-mon[41371]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:21.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:21 smithi146 ceph-mon[33158]: from='osd.7 v1:172.21.15.146:6812/2181809960' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:00:21.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:21 smithi146 ceph-mon[33158]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:00:21.812 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:21.810+0000 7f67317fa640 1 -- 172.21.15.146:0/151905749 <== mgr.14150 v1:172.21.15.102:6800/2950771867 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 529969348) 0x7f6718000bc0 con 0x7f672406f760 2024-09-11T08:00:21.812 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 7 on host 'smithi146' 2024-09-11T08:00:21.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:21.812+0000 7f673ab0d640 1 -- 172.21.15.146:0/151905749 >> v1:172.21.15.102:6800/2950771867 conn(0x7f672406f760 legacy=0x7f6724071c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:21.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:21.812+0000 7f673ab0d640 1 -- 172.21.15.146:0/151905749 >> v1:172.21.15.146:6789/0 conn(0x7f67340fc510 legacy=0x7f6734071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:21.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:21.812+0000 7f673ab0d640 1 -- 172.21.15.146:0/151905749 shutdown_connections 2024-09-11T08:00:21.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:21.812+0000 7f673ab0d640 1 -- 172.21.15.146:0/151905749 >> 172.21.15.146:0/151905749 conn(0x7f67340f7cd0 msgr2=0x7f6734104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:00:21.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:21.812+0000 7f673ab0d640 1 -- 172.21.15.146:0/151905749 shutdown_connections 2024-09-11T08:00:21.814 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:21.812+0000 7f673ab0d640 1 -- 172.21.15.146:0/151905749 wait complete. 2024-09-11T08:00:21.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:21 smithi102 ceph-mon[29849]: from='osd.7 v1:172.21.15.146:6812/2181809960' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:00:21.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:21 smithi102 ceph-mon[29849]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:00:21.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:21 smithi102 ceph-mon[41371]: from='osd.7 v1:172.21.15.146:6812/2181809960' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:00:21.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:21 smithi102 ceph-mon[41371]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T08:00:22.411 DEBUG:teuthology.orchestra.run.smithi146:osd.7> sudo journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.7.service 2024-09-11T08:00:22.414 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-11T08:00:22.414 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T08:00:22.684 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:00:22 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:00:22.531+0000 7f6314bda640 -1 osd.7 0 waiting for initial osdmap 2024-09-11T08:00:22.684 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:00:22 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:00:22.537+0000 7f630c9e9640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T08:00:22.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:22 smithi146 ceph-mon[33158]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:22.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:22 smithi146 ceph-mon[33158]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T08:00:22.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:22 smithi146 ceph-mon[33158]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T08:00:22.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:22 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:00:22.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:22 smithi146 ceph-mon[33158]: from='osd.7 v1:172.21.15.146:6812/2181809960' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T08:00:22.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:22 smithi146 ceph-mon[33158]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T08:00:22.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:22 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:22.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:22 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:22.686 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:00:22.810 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:22 smithi102 ceph-mon[29849]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:22.810 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:22 smithi102 ceph-mon[29849]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T08:00:22.810 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:22 smithi102 ceph-mon[29849]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T08:00:22.810 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:22 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:00:22.810 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:22 smithi102 ceph-mon[29849]: from='osd.7 v1:172.21.15.146:6812/2181809960' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T08:00:22.810 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:22 smithi102 ceph-mon[29849]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T08:00:22.810 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:22 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:22.810 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:22 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:22.811 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:22 smithi102 ceph-mon[41371]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:22.811 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:22 smithi102 ceph-mon[41371]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T08:00:22.811 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:22 smithi102 ceph-mon[41371]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T08:00:22.811 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:22 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:00:22.811 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:22 smithi102 ceph-mon[41371]: from='osd.7 v1:172.21.15.146:6812/2181809960' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T08:00:22.811 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:22 smithi102 ceph-mon[41371]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-11T08:00:22.811 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:22 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:22.811 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:22 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:23.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:23 smithi146 ceph-mon[33158]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-11T08:00:23.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:23 smithi146 ceph-mon[33158]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T08:00:23.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:23 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:00:23.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:23 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:00:23.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:23 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:23.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:23 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:23.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:23 smithi102 ceph-mon[29849]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-11T08:00:23.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:23 smithi102 ceph-mon[29849]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T08:00:23.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:23 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:00:23.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:23 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:00:23.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:23 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:23.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:23 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:23.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:23 smithi102 ceph-mon[41371]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-11T08:00:23.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:23 smithi102 ceph-mon[41371]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T08:00:23.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:23 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:00:23.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:23 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:00:23.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:23 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:23.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:23 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:24.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.360+0000 7f5c055c1640 1 -- 172.21.15.102:0/3976160432 >> v1:172.21.15.102:6790/0 conn(0x7f5c000fb490 legacy=0x7f5c000fb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:24.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.360+0000 7f5c055c1640 1 -- 172.21.15.102:0/3976160432 shutdown_connections 2024-09-11T08:00:24.362 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.360+0000 7f5c055c1640 1 -- 172.21.15.102:0/3976160432 >> 172.21.15.102:0/3976160432 conn(0x7f5c000f6c30 msgr2=0x7f5c000f9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:00:24.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.360+0000 7f5c055c1640 1 -- 172.21.15.102:0/3976160432 shutdown_connections 2024-09-11T08:00:24.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.360+0000 7f5c055c1640 1 -- 172.21.15.102:0/3976160432 wait complete. 2024-09-11T08:00:24.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.361+0000 7f5c055c1640 1 Processor -- start 2024-09-11T08:00:24.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.361+0000 7f5c055c1640 1 -- start start 2024-09-11T08:00:24.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.361+0000 7f5c055c1640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5c00119800 con 0x7f5c000ff990 2024-09-11T08:00:24.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.361+0000 7f5c055c1640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5c0011aa00 con 0x7f5c000fb490 2024-09-11T08:00:24.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.361+0000 7f5c055c1640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5c0011bc00 con 0x7f5c001035b0 2024-09-11T08:00:24.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.361+0000 7f5bfe7fc640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f5c000ff990 0x7f5c001147b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47174/0 (socket says 172.21.15.102:47174) 2024-09-11T08:00:24.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.361+0000 7f5bfe7fc640 1 -- 172.21.15.102:0/495676486 learned_addr learned my addr 172.21.15.102:0/495676486 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:00:24.363 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.361+0000 7f5bff7fe640 1 --1- 172.21.15.102:0/495676486 >> v1:172.21.15.102:6790/0 conn(0x7f5c001035b0 0x7f5c00117f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:53692/0 (socket says 172.21.15.102:53692) 2024-09-11T08:00:24.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.362+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559777280 0 0) 0x7f5c00119800 con 0x7f5c000ff990 2024-09-11T08:00:24.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.362+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5be80034e0 con 0x7f5c000ff990 2024-09-11T08:00:24.365 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.362+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 599712731 0 0) 0x7f5c0011bc00 con 0x7f5c001035b0 2024-09-11T08:00:24.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.362+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c00119800 con 0x7f5c001035b0 2024-09-11T08:00:24.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.362+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 434071538 0 0) 0x7f5c0011aa00 con 0x7f5c000fb490 2024-09-11T08:00:24.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.362+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c0011bc00 con 0x7f5c000fb490 2024-09-11T08:00:24.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.362+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4143989092 0 0) 0x7f5c00119800 con 0x7f5c001035b0 2024-09-11T08:00:24.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.362+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5c0011aa00 con 0x7f5c001035b0 2024-09-11T08:00:24.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.362+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f5bf8001c80 con 0x7f5c001035b0 2024-09-11T08:00:24.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.362+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2231412858 0 0) 0x7f5be80034e0 con 0x7f5c000ff990 2024-09-11T08:00:24.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.362+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5c00119800 con 0x7f5c000ff990 2024-09-11T08:00:24.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.362+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f5bf4001900 con 0x7f5c000ff990 2024-09-11T08:00:24.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.362+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 192152333 0 0) 0x7f5c0011bc00 con 0x7f5c000fb490 2024-09-11T08:00:24.366 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.362+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5be80034e0 con 0x7f5c000fb490 2024-09-11T08:00:24.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.362+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f5bf0002010 con 0x7f5c000fb490 2024-09-11T08:00:24.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.362+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2069013109 0 0) 0x7f5c0011aa00 con 0x7f5c001035b0 2024-09-11T08:00:24.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.362+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 >> v1:172.21.15.146:6789/0 conn(0x7f5c000fb490 legacy=0x7f5c00066080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:24.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.362+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 >> v1:172.21.15.102:6789/0 conn(0x7f5c000ff990 legacy=0x7f5c001147b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:24.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.362+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0011ce00 con 0x7f5c001035b0 2024-09-11T08:00:24.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.362+0000 7f5c055c1640 1 -- 172.21.15.102:0/495676486 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c0011bdf0 con 0x7f5c001035b0 2024-09-11T08:00:24.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.362+0000 7f5c055c1640 1 -- 172.21.15.102:0/495676486 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c0011c240 con 0x7f5c001035b0 2024-09-11T08:00:24.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.362+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f5bf80025c0 con 0x7f5c001035b0 2024-09-11T08:00:24.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.362+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f5bf8005c50 con 0x7f5c001035b0 2024-09-11T08:00:24.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.363+0000 7f5c055c1640 1 -- 172.21.15.102:0/495676486 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bec005040 con 0x7f5c001035b0 2024-09-11T08:00:24.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.364+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 1029906827 0 0) 0x7f5bf8002170 con 0x7f5c001035b0 2024-09-11T08:00:24.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.364+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(48..48 src has 1..48) v4 ==== 4110+0+0 (unknown 2376648383 0 0) 0x7f5bf8089690 con 0x7f5c001035b0 2024-09-11T08:00:24.369 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.368+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f5bf8056be0 con 0x7f5c001035b0 2024-09-11T08:00:24.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.524+0000 7f5c055c1640 1 -- 172.21.15.102:0/495676486 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f5bec0052f0 con 0x7f5c001035b0 2024-09-11T08:00:24.525 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.524+0000 7f5be77fe640 1 -- 172.21.15.102:0/495676486 <== mon.2 v1:172.21.15.102:6790/0 10 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v48) v1 ==== 74+0+130 (unknown 4182864030 0 4171077386) 0x7f5bf8056580 con 0x7f5c001035b0 2024-09-11T08:00:24.526 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T08:00:24.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.526+0000 7f5c055c1640 1 -- 172.21.15.102:0/495676486 >> v1:172.21.15.102:6800/2950771867 conn(0x7f5be806fc40 legacy=0x7f5be80720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:24.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.526+0000 7f5c055c1640 1 -- 172.21.15.102:0/495676486 >> v1:172.21.15.102:6790/0 conn(0x7f5c001035b0 legacy=0x7f5c00117f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:24.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.526+0000 7f5c055c1640 1 -- 172.21.15.102:0/495676486 shutdown_connections 2024-09-11T08:00:24.527 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.526+0000 7f5c055c1640 1 -- 172.21.15.102:0/495676486 >> 172.21.15.102:0/495676486 conn(0x7f5c000f6c30 msgr2=0x7f5c00105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:00:24.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.526+0000 7f5c055c1640 1 -- 172.21.15.102:0/495676486 shutdown_connections 2024-09-11T08:00:24.528 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:24.526+0000 7f5c055c1640 1 -- 172.21.15.102:0/495676486 wait complete. 2024-09-11T08:00:24.641 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:24 smithi146 ceph-mon[33158]: purged_snaps scrub starts 2024-09-11T08:00:24.641 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:24 smithi146 ceph-mon[33158]: purged_snaps scrub ok 2024-09-11T08:00:24.641 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:24 smithi146 ceph-mon[33158]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:24.641 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:24 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:00:24.641 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:24 smithi146 ceph-mon[33158]: osd.7 v1:172.21.15.146:6812/2181809960 boot 2024-09-11T08:00:24.641 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:24 smithi146 ceph-mon[33158]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T08:00:24.641 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:24 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:00:24.641 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:24 smithi146 ceph-mon[33158]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:00:24.641 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/495676486' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T08:00:24.814 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:24 smithi102 ceph-mon[29849]: purged_snaps scrub starts 2024-09-11T08:00:24.814 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:24 smithi102 ceph-mon[29849]: purged_snaps scrub ok 2024-09-11T08:00:24.814 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:24 smithi102 ceph-mon[29849]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:24.814 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:24 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:00:24.814 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:24 smithi102 ceph-mon[29849]: osd.7 v1:172.21.15.146:6812/2181809960 boot 2024-09-11T08:00:24.814 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:24 smithi102 ceph-mon[29849]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T08:00:24.815 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:24 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:00:24.815 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:24 smithi102 ceph-mon[29849]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:00:24.815 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/495676486' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T08:00:24.815 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:24 smithi102 ceph-mon[41371]: purged_snaps scrub starts 2024-09-11T08:00:24.815 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:24 smithi102 ceph-mon[41371]: purged_snaps scrub ok 2024-09-11T08:00:24.816 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:24 smithi102 ceph-mon[41371]: pgmap v157: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T08:00:24.816 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:24 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:00:24.816 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:24 smithi102 ceph-mon[41371]: osd.7 v1:172.21.15.146:6812/2181809960 boot 2024-09-11T08:00:24.816 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:24 smithi102 ceph-mon[41371]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T08:00:24.816 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:24 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:00:24.816 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:24 smithi102 ceph-mon[41371]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T08:00:24.816 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/495676486' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T08:00:25.281 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":48,"num_osds":8,"num_up_osds":8,"osd_up_since":1726041623,"num_in_osds":8,"osd_in_since":1726041607,"num_remapped_pgs":1} 2024-09-11T08:00:25.282 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T08:00:25.525 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:00:25.657 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:25 smithi102 ceph-mon[29849]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T08:00:25.658 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:25 smithi102 ceph-mon[41371]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T08:00:25.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:25 smithi146 ceph-mon[33158]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T08:00:26.814 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[41371]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:00:26.814 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[41371]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T08:00:26.814 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:26.814 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:26.814 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:00:26.814 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:00:26.814 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:00:26.814 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:00:26.814 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:26.814 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:26.814 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:00:26.815 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:26.815 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:00:26.815 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:26.815 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:00:26.815 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:26.815 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[29849]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:00:26.815 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[29849]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T08:00:26.815 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:26.815 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:26.816 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:00:26.816 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:00:26.816 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:00:26.816 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:00:26.816 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:26.816 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:26.816 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:00:26.816 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:26.816 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:00:26.816 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:26.816 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:00:26.816 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:26 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:26.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:26 smithi146 ceph-mon[33158]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:00:26.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:26 smithi146 ceph-mon[33158]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T08:00:26.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:26.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:26.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:00:26.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:00:26.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:00:26.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:00:26.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:26.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:26.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:00:26.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:26.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:00:26.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:26.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:00:26.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:26 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:27.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.126+0000 7ff3572f3640 1 -- 172.21.15.102:0/2940173870 >> v1:172.21.15.102:6789/0 conn(0x7ff3501042a0 legacy=0x7ff350106680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:27.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.126+0000 7ff3572f3640 1 -- 172.21.15.102:0/2940173870 shutdown_connections 2024-09-11T08:00:27.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.126+0000 7ff3572f3640 1 -- 172.21.15.102:0/2940173870 >> 172.21.15.102:0/2940173870 conn(0x7ff3500752c0 msgr2=0x7ff3500f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:00:27.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.126+0000 7ff3572f3640 1 -- 172.21.15.102:0/2940173870 shutdown_connections 2024-09-11T08:00:27.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.126+0000 7ff3572f3640 1 -- 172.21.15.102:0/2940173870 wait complete. 2024-09-11T08:00:27.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.127+0000 7ff3572f3640 1 Processor -- start 2024-09-11T08:00:27.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.127+0000 7ff3572f3640 1 -- start start 2024-09-11T08:00:27.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.127+0000 7ff3572f3640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff35011df70 con 0x7ff3501042a0 2024-09-11T08:00:27.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.127+0000 7ff3572f3640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff35011f110 con 0x7ff350107e00 2024-09-11T08:00:27.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.127+0000 7ff3572f3640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff3501202b0 con 0x7ff350101530 2024-09-11T08:00:27.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.127+0000 7ff35508f640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7ff350101530 0x7ff350108300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:53708/0 (socket says 172.21.15.102:53708) 2024-09-11T08:00:27.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.127+0000 7ff35508f640 1 -- 172.21.15.102:0/2081359328 learned_addr learned my addr 172.21.15.102:0/2081359328 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:00:27.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.127+0000 7ff35488e640 1 --1- 172.21.15.102:0/2081359328 >> v1:172.21.15.102:6789/0 conn(0x7ff3501042a0 0x7ff350108a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47196/0 (socket says 172.21.15.102:47196) 2024-09-11T08:00:27.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.127+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 895464888 0 0) 0x7ff35011df70 con 0x7ff3501042a0 2024-09-11T08:00:27.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.127+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff33c0034e0 con 0x7ff3501042a0 2024-09-11T08:00:27.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.127+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2166037298 0 0) 0x7ff3501202b0 con 0x7ff350101530 2024-09-11T08:00:27.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.127+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff35011df70 con 0x7ff350101530 2024-09-11T08:00:27.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.128+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3137102481 0 0) 0x7ff35011f110 con 0x7ff350107e00 2024-09-11T08:00:27.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.128+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3501202b0 con 0x7ff350107e00 2024-09-11T08:00:27.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.128+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 889157859 0 0) 0x7ff33c0034e0 con 0x7ff3501042a0 2024-09-11T08:00:27.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.128+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff35011f110 con 0x7ff3501042a0 2024-09-11T08:00:27.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.128+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7ff340001360 con 0x7ff3501042a0 2024-09-11T08:00:27.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.128+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3467618168 0 0) 0x7ff35011df70 con 0x7ff350101530 2024-09-11T08:00:27.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.128+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff33c0034e0 con 0x7ff350101530 2024-09-11T08:00:27.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.128+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3300167125 0 0) 0x7ff3501202b0 con 0x7ff350107e00 2024-09-11T08:00:27.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.128+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff35011df70 con 0x7ff350107e00 2024-09-11T08:00:27.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.128+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7ff34c0019f0 con 0x7ff350101530 2024-09-11T08:00:27.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.128+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7ff348001b40 con 0x7ff350107e00 2024-09-11T08:00:27.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.128+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1211229738 0 0) 0x7ff35011f110 con 0x7ff3501042a0 2024-09-11T08:00:27.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.128+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 >> v1:172.21.15.102:6790/0 conn(0x7ff350101530 legacy=0x7ff350108300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:27.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.128+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 >> v1:172.21.15.146:6789/0 conn(0x7ff350107e00 legacy=0x7ff35011c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:27.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.128+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3501214d0 con 0x7ff3501042a0 2024-09-11T08:00:27.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.128+0000 7ff3572f3640 1 -- 172.21.15.102:0/2081359328 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3501204a0 con 0x7ff3501042a0 2024-09-11T08:00:27.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.128+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7ff3400015a0 con 0x7ff3501042a0 2024-09-11T08:00:27.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.128+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7ff3400036a0 con 0x7ff3501042a0 2024-09-11T08:00:27.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.128+0000 7ff3572f3640 1 -- 172.21.15.102:0/2081359328 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff350120970 con 0x7ff3501042a0 2024-09-11T08:00:27.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.129+0000 7ff3572f3640 1 -- 172.21.15.102:0/2081359328 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3500fcc30 con 0x7ff3501042a0 2024-09-11T08:00:27.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.129+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 1029906827 0 0) 0x7ff340003800 con 0x7ff3501042a0 2024-09-11T08:00:27.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.130+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(50..50 src has 1..50) v4 ==== 4077+0+0 (unknown 1200487953 0 0) 0x7ff340089700 con 0x7ff3501042a0 2024-09-11T08:00:27.136 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.135+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7ff340056c70 con 0x7ff3501042a0 2024-09-11T08:00:27.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.289+0000 7ff3572f3640 1 -- 172.21.15.102:0/2081359328 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff35010d880 con 0x7ff3501042a0 2024-09-11T08:00:27.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.290+0000 7ff345ffb640 1 -- 172.21.15.102:0/2081359328 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+11182 (unknown 1916093452 0 2179469009) 0x7ff340056610 con 0x7ff3501042a0 2024-09-11T08:00:27.292 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T08:00:27.292 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":50,"fsid":"2051892c-7013-11ef-bcea-c7b262605968","created":"2024-09-11T07:55:31.241924+0000","modified":"2024-09-11T08:00:25.536303+0000","last_up_change":"2024-09-11T08:00:23.529399+0000","last_in_change":"2024-09-11T08:00:07.275908+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T07:58:29.695819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"08248393-1eca-4c59-b577-e0ee4ed4d6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":444496098}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":444496098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":444496098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":444496098}]},"public_addr":"172.21.15.102:6801/444496098","cluster_addr":"172.21.15.102:6802/444496098","heartbeat_back_addr":"172.21.15.102:6804/444496098","heartbeat_front_addr":"172.21.15.102:6803/444496098","state":["exists","up"]},{"osd":1,"uuid":"209cc4e5-6ff0-49f4-8ce4-0540ea1b5c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":2501991548}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":2501991548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":2501991548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":2501991548}]},"public_addr":"172.21.15.102:6805/2501991548","cluster_addr":"172.21.15.102:6806/2501991548","heartbeat_back_addr":"172.21.15.102:6808/2501991548","heartbeat_front_addr":"172.21.15.102:6807/2501991548","state":["exists","up"]},{"osd":2,"uuid":"e900567f-3dca-4cde-abbb-d8fa53cec635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":3842496257}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":3842496257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":3842496257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":3842496257}]},"public_addr":"172.21.15.102:6809/3842496257","cluster_addr":"172.21.15.102:6810/3842496257","heartbeat_back_addr":"172.21.15.102:6812/3842496257","heartbeat_front_addr":"172.21.15.102:6811/3842496257","state":["exists","up"]},{"osd":3,"uuid":"c0d77f9c-0e1b-45b5-8b9b-e84097814fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":426874071}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":426874071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6816","nonce":426874071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":426874071}]},"public_addr":"172.21.15.102:6813/426874071","cluster_addr":"172.21.15.102:6814/426874071","heartbeat_back_addr":"172.21.15.102:6816/426874071","heartbeat_front_addr":"172.21.15.102:6815/426874071","state":["exists","up"]},{"osd":4,"uuid":"31d92bc5-10be-47d0-b91c-5e5496c25ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6800","nonce":2908355826}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6801","nonce":2908355826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6803","nonce":2908355826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6802","nonce":2908355826}]},"public_addr":"172.21.15.146:6800/2908355826","cluster_addr":"172.21.15.146:6801/2908355826","heartbeat_back_addr":"172.21.15.146:6803/2908355826","heartbeat_front_addr":"172.21.15.146:6802/2908355826","state":["exists","up"]},{"osd":5,"uuid":"6272822a-2ca3-45c8-a3c6-c00e26753989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6804","nonce":1497668584}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6805","nonce":1497668584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6807","nonce":1497668584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6806","nonce":1497668584}]},"public_addr":"172.21.15.146:6804/1497668584","cluster_addr":"172.21.15.146:6805/1497668584","heartbeat_back_addr":"172.21.15.146:6807/1497668584","heartbeat_front_addr":"172.21.15.146:6806/1497668584","state":["exists","up"]},{"osd":6,"uuid":"1c3a857f-a8d5-46d0-8fd5-bcc759074f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6808","nonce":85686736}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6809","nonce":85686736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6811","nonce":85686736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6810","nonce":85686736}]},"public_addr":"172.21.15.146:6808/85686736","cluster_addr":"172.21.15.146:6809/85686736","heartbeat_back_addr":"172.21.15.146:6811/85686736","heartbeat_front_addr":"172.21.15.146:6810/85686736","state":["exists","up"]},{"osd":7,"uuid":"ca7e36e9-ca16-4d15-8264-deed759fdbf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6812","nonce":2181809960}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6813","nonce":2181809960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6815","nonce":2181809960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6814","nonce":2181809960}]},"public_addr":"172.21.15.146:6812/2181809960","cluster_addr":"172.21.15.146:6813/2181809960","heartbeat_back_addr":"172.21.15.146:6815/2181809960","heartbeat_front_addr":"172.21.15.146:6814/2181809960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:57:40.079569+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:58:03.521245+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:58:27.496621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:58:50.615763+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:59:11.931460+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:59:35.063555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:59:58.367568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:00:22.451943+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:0/2854313951":"2024-09-12T07:56:23.399794+0000","172.21.15.102:0/4190790614":"2024-09-12T07:56:23.399794+0000","172.21.15.102:0/3435847265":"2024-09-12T07:56:23.399794+0000","172.21.15.102:0/234741464":"2024-09-12T07:56:01.048720+0000","172.21.15.102:6800/3158655170":"2024-09-12T07:56:01.048720+0000","172.21.15.102:6800/3294079105":"2024-09-12T07:56:23.399794+0000","172.21.15.102:0/3790980491":"2024-09-12T07:56:01.048720+0000","172.21.15.102:0/831066729":"2024-09-12T07:56:01.048720+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:00:27.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.292+0000 7ff3572f3640 1 -- 172.21.15.102:0/2081359328 >> v1:172.21.15.102:6800/2950771867 conn(0x7ff33c070040 legacy=0x7ff33c0724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:27.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.292+0000 7ff3572f3640 1 -- 172.21.15.102:0/2081359328 >> v1:172.21.15.102:6789/0 conn(0x7ff3501042a0 legacy=0x7ff350108a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:27.293 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.292+0000 7ff3572f3640 1 -- 172.21.15.102:0/2081359328 shutdown_connections 2024-09-11T08:00:27.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.292+0000 7ff3572f3640 1 -- 172.21.15.102:0/2081359328 >> 172.21.15.102:0/2081359328 conn(0x7ff3500752c0 msgr2=0x7ff350064e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:00:27.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.292+0000 7ff3572f3640 1 -- 172.21.15.102:0/2081359328 shutdown_connections 2024-09-11T08:00:27.294 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:27.292+0000 7ff3572f3640 1 -- 172.21.15.102:0/2081359328 wait complete. 2024-09-11T08:00:27.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:27 smithi102 ceph-mon[29849]: Detected new or changed devices on smithi146 2024-09-11T08:00:27.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:27 smithi102 ceph-mon[29849]: Adjusting osd_memory_target on smithi146 to 4272M 2024-09-11T08:00:27.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2081359328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:00:27.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:27 smithi102 ceph-mon[41371]: Detected new or changed devices on smithi146 2024-09-11T08:00:27.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:27 smithi102 ceph-mon[41371]: Adjusting osd_memory_target on smithi146 to 4272M 2024-09-11T08:00:27.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2081359328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:00:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:27 smithi146 ceph-mon[33158]: Detected new or changed devices on smithi146 2024-09-11T08:00:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:27 smithi146 ceph-mon[33158]: Adjusting osd_memory_target on smithi146 to 4272M 2024-09-11T08:00:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2081359328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:00:27.935 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T07:58:29.695819+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-09-11T08:00:27.935 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-11T08:00:28.176 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:00:28.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:28 smithi102 ceph-mon[29849]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:00:28.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:28 smithi102 ceph-mon[41371]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:00:28.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:28 smithi146 ceph-mon[33158]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:00:29.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.928+0000 7fe4ec975640 1 -- 172.21.15.102:0/4085944839 >> v1:172.21.15.102:6790/0 conn(0x7fe4e8109130 legacy=0x7fe4e810b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:29.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.928+0000 7fe4ec975640 1 -- 172.21.15.102:0/4085944839 shutdown_connections 2024-09-11T08:00:29.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.928+0000 7fe4ec975640 1 -- 172.21.15.102:0/4085944839 >> 172.21.15.102:0/4085944839 conn(0x7fe4e8074cb0 msgr2=0x7fe4e80f8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:00:29.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.928+0000 7fe4ec975640 1 -- 172.21.15.102:0/4085944839 shutdown_connections 2024-09-11T08:00:29.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.929+0000 7fe4ec975640 1 -- 172.21.15.102:0/4085944839 wait complete. 2024-09-11T08:00:29.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.929+0000 7fe4ec975640 1 Processor -- start 2024-09-11T08:00:29.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.929+0000 7fe4ec975640 1 -- start start 2024-09-11T08:00:29.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.929+0000 7fe4ec975640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe4e8101d60 con 0x7fe4e8109130 2024-09-11T08:00:29.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.929+0000 7fe4ec975640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe4e8101ef0 con 0x7fe4e8105570 2024-09-11T08:00:29.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.929+0000 7fe4ec975640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe4e81020c0 con 0x7fe4e8102840 2024-09-11T08:00:29.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.929+0000 7fe4e659c640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7fe4e8102840 0x7fe4e81015c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:53654/0 (socket says 172.21.15.102:53654) 2024-09-11T08:00:29.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.929+0000 7fe4e659c640 1 -- 172.21.15.102:0/975049948 learned_addr learned my addr 172.21.15.102:0/975049948 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:00:29.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.930+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2855580611 0 0) 0x7fe4e81020c0 con 0x7fe4e8102840 2024-09-11T08:00:29.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.930+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4d00034e0 con 0x7fe4e8102840 2024-09-11T08:00:29.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.930+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 528905420 0 0) 0x7fe4e8101d60 con 0x7fe4e8109130 2024-09-11T08:00:29.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.930+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4e81020c0 con 0x7fe4e8109130 2024-09-11T08:00:29.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.930+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112858688 0 0) 0x7fe4e8101ef0 con 0x7fe4e8105570 2024-09-11T08:00:29.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.930+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4e8101d60 con 0x7fe4e8105570 2024-09-11T08:00:29.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.930+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 702232259 0 0) 0x7fe4d00034e0 con 0x7fe4e8102840 2024-09-11T08:00:29.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.930+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe4e8101ef0 con 0x7fe4e8102840 2024-09-11T08:00:29.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.930+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4129281450 0 0) 0x7fe4e81020c0 con 0x7fe4e8109130 2024-09-11T08:00:29.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.930+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe4d00034e0 con 0x7fe4e8109130 2024-09-11T08:00:29.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.930+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fe4d8001940 con 0x7fe4e8102840 2024-09-11T08:00:29.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.930+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fe4e0001360 con 0x7fe4e8109130 2024-09-11T08:00:29.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.930+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1945442432 0 0) 0x7fe4e8101d60 con 0x7fe4e8105570 2024-09-11T08:00:29.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.930+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe4e81020c0 con 0x7fe4e8105570 2024-09-11T08:00:29.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.930+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fe4dc0016a0 con 0x7fe4e8105570 2024-09-11T08:00:29.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.930+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3151063262 0 0) 0x7fe4e8101ef0 con 0x7fe4e8102840 2024-09-11T08:00:29.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.930+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 >> v1:172.21.15.146:6789/0 conn(0x7fe4e8105570 legacy=0x7fe4e8119b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:29.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.930+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 >> v1:172.21.15.102:6789/0 conn(0x7fe4e8109130 legacy=0x7fe4e811d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:29.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.930+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e81219b0 con 0x7fe4e8102840 2024-09-11T08:00:29.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.930+0000 7fe4ec975640 1 -- 172.21.15.102:0/975049948 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e811e9e0 con 0x7fe4e8102840 2024-09-11T08:00:29.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.930+0000 7fe4ec975640 1 -- 172.21.15.102:0/975049948 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e811eeb0 con 0x7fe4e8102840 2024-09-11T08:00:29.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.930+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7fe4d8001ca0 con 0x7fe4e8102840 2024-09-11T08:00:29.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.930+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fe4d80059a0 con 0x7fe4e8102840 2024-09-11T08:00:29.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.931+0000 7fe4ec975640 1 -- 172.21.15.102:0/975049948 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c8005040 con 0x7fe4e8102840 2024-09-11T08:00:29.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.932+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 1029906827 0 0) 0x7fe4d8002230 con 0x7fe4e8102840 2024-09-11T08:00:29.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.932+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(50..50 src has 1..50) v4 ==== 4077+0+0 (unknown 1200487953 0 0) 0x7fe4d8089590 con 0x7fe4e8102840 2024-09-11T08:00:29.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:29.936+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fe4d8056aa0 con 0x7fe4e8102840 2024-09-11T08:00:30.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:30.090+0000 7fe4ec975640 1 -- 172.21.15.102:0/975049948 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe4c8005870 con 0x7fe4e8102840 2024-09-11T08:00:30.091 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:30.090+0000 7fe4d6ffd640 1 -- 172.21.15.102:0/975049948 <== mon.2 v1:172.21.15.102:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (unknown 3046549997 0 2170607528) 0x7fe4d8056440 con 0x7fe4e8102840 2024-09-11T08:00:30.092 INFO:teuthology.orchestra.run.smithi102.stdout:pg_num: 1 2024-09-11T08:00:30.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:30.093+0000 7fe4ec975640 1 -- 172.21.15.102:0/975049948 >> v1:172.21.15.102:6800/2950771867 conn(0x7fe4d006fd80 legacy=0x7fe4d0072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:30.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:30.093+0000 7fe4ec975640 1 -- 172.21.15.102:0/975049948 >> v1:172.21.15.102:6790/0 conn(0x7fe4e8102840 legacy=0x7fe4e81015c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:30.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:30.093+0000 7fe4ec975640 1 -- 172.21.15.102:0/975049948 shutdown_connections 2024-09-11T08:00:30.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:30.093+0000 7fe4ec975640 1 -- 172.21.15.102:0/975049948 >> 172.21.15.102:0/975049948 conn(0x7fe4e8074cb0 msgr2=0x7fe4e80fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:00:30.094 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:30.093+0000 7fe4ec975640 1 -- 172.21.15.102:0/975049948 shutdown_connections 2024-09-11T08:00:30.095 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:00:30.093+0000 7fe4ec975640 1 -- 172.21.15.102:0/975049948 wait complete. 2024-09-11T08:00:30.615 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi102 2024-09-11T08:00:30.615 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi102=foo.a' 2024-09-11T08:00:30.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:30 smithi102 ceph-mon[41371]: pgmap v163: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:00:30.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:30 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/975049948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T08:00:30.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:30 smithi102 ceph-mon[29849]: pgmap v163: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:00:30.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:30 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/975049948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T08:00:30.871 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:00:30.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:30 smithi146 ceph-mon[33158]: pgmap v163: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:00:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:30 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/975049948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T08:00:31.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:31 smithi102 ceph-mon[41371]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:00:31.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:31 smithi102 ceph-mon[41371]: Cluster is now healthy 2024-09-11T08:00:31.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:31 smithi102 ceph-mon[29849]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:00:31.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:31 smithi102 ceph-mon[29849]: Cluster is now healthy 2024-09-11T08:00:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:31 smithi146 ceph-mon[33158]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T08:00:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:31 smithi146 ceph-mon[33158]: Cluster is now healthy 2024-09-11T08:00:32.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.584+0000 7f1c50fb5640 1 -- 172.21.15.146:0/2448768040 >> v1:172.21.15.146:6789/0 conn(0x7f1c4c1035a0 legacy=0x7f1c4c105a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:32.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.584+0000 7f1c50fb5640 1 -- 172.21.15.146:0/2448768040 shutdown_connections 2024-09-11T08:00:32.593 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.584+0000 7f1c50fb5640 1 -- 172.21.15.146:0/2448768040 >> 172.21.15.146:0/2448768040 conn(0x7f1c4c0f6c20 msgr2=0x7f1c4c0f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:00:32.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.584+0000 7f1c50fb5640 1 -- 172.21.15.146:0/2448768040 shutdown_connections 2024-09-11T08:00:32.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.584+0000 7f1c50fb5640 1 -- 172.21.15.146:0/2448768040 wait complete. 2024-09-11T08:00:32.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.585+0000 7f1c50fb5640 1 Processor -- start 2024-09-11T08:00:32.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.585+0000 7f1c50fb5640 1 -- start start 2024-09-11T08:00:32.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.585+0000 7f1c50fb5640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1c4c117790 con 0x7f1c4c1035a0 2024-09-11T08:00:32.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.585+0000 7f1c50fb5640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1c4c118990 con 0x7f1c4c0fb480 2024-09-11T08:00:32.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.585+0000 7f1c50fb5640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1c4c119b90 con 0x7f1c4c0ff980 2024-09-11T08:00:32.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.585+0000 7f1c4a59c640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f1c4c0fb480 0x7f1c4c065840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:43262/0 (socket says 172.21.15.146:43262) 2024-09-11T08:00:32.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.585+0000 7f1c4a59c640 1 -- 172.21.15.146:0/572573913 learned_addr learned my addr 172.21.15.146:0/572573913 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-11T08:00:32.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.585+0000 7f1c3affd640 1 -- 172.21.15.146:0/572573913 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2650380678 0 0) 0x7f1c4c118990 con 0x7f1c4c0fb480 2024-09-11T08:00:32.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.585+0000 7f1c3affd640 1 -- 172.21.15.146:0/572573913 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c34003120 con 0x7f1c4c0fb480 2024-09-11T08:00:32.594 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.585+0000 7f1c3affd640 1 -- 172.21.15.146:0/572573913 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249433539 0 0) 0x7f1c4c119b90 con 0x7f1c4c0ff980 2024-09-11T08:00:32.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.585+0000 7f1c3affd640 1 -- 172.21.15.146:0/572573913 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c4c118990 con 0x7f1c4c0ff980 2024-09-11T08:00:32.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.585+0000 7f1c3affd640 1 -- 172.21.15.146:0/572573913 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3054597267 0 0) 0x7f1c4c117790 con 0x7f1c4c1035a0 2024-09-11T08:00:32.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.585+0000 7f1c3affd640 1 -- 172.21.15.146:0/572573913 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c4c119b90 con 0x7f1c4c1035a0 2024-09-11T08:00:32.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.585+0000 7f1c3affd640 1 -- 172.21.15.146:0/572573913 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3682355479 0 0) 0x7f1c34003120 con 0x7f1c4c0fb480 2024-09-11T08:00:32.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.585+0000 7f1c3affd640 1 -- 172.21.15.146:0/572573913 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1c4c117790 con 0x7f1c4c0fb480 2024-09-11T08:00:32.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.585+0000 7f1c3affd640 1 -- 172.21.15.146:0/572573913 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f1c3c001940 con 0x7f1c4c0fb480 2024-09-11T08:00:32.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.585+0000 7f1c3affd640 1 -- 172.21.15.146:0/572573913 <== mon.1 v1:172.21.15.146:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1957928100 0 0) 0x7f1c4c117790 con 0x7f1c4c0fb480 2024-09-11T08:00:32.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.585+0000 7f1c3affd640 1 -- 172.21.15.146:0/572573913 >> v1:172.21.15.102:6790/0 conn(0x7f1c4c0ff980 legacy=0x7f1c4c1124e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:32.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.585+0000 7f1c3affd640 1 -- 172.21.15.146:0/572573913 >> v1:172.21.15.102:6789/0 conn(0x7f1c4c1035a0 legacy=0x7f1c4c115e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:32.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.585+0000 7f1c3affd640 1 -- 172.21.15.146:0/572573913 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c11ad90 con 0x7f1c4c0fb480 2024-09-11T08:00:32.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.586+0000 7f1c50fb5640 1 -- 172.21.15.146:0/572573913 --> v1:172.21.15.146:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4c119d80 con 0x7f1c4c0fb480 2024-09-11T08:00:32.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.586+0000 7f1c50fb5640 1 -- 172.21.15.146:0/572573913 --> v1:172.21.15.146:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4c11a1d0 con 0x7f1c4c0fb480 2024-09-11T08:00:32.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.586+0000 7f1c3affd640 1 -- 172.21.15.146:0/572573913 <== mon.1 v1:172.21.15.146:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f1c3c001cd0 con 0x7f1c4c0fb480 2024-09-11T08:00:32.595 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.586+0000 7f1c3affd640 1 -- 172.21.15.146:0/572573913 <== mon.1 v1:172.21.15.146:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f1c3c005b60 con 0x7f1c4c0fb480 2024-09-11T08:00:32.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.587+0000 7f1c50fb5640 1 -- 172.21.15.146:0/572573913 --> v1:172.21.15.146:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c2c005040 con 0x7f1c4c0fb480 2024-09-11T08:00:32.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.587+0000 7f1c3affd640 1 -- 172.21.15.146:0/572573913 <== mon.1 v1:172.21.15.146:6789/0 7 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 1029906827 0 0) 0x7f1c3c01bf50 con 0x7f1c4c0fb480 2024-09-11T08:00:32.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.588+0000 7f1c3affd640 1 -- 172.21.15.146:0/572573913 <== mon.1 v1:172.21.15.146:6789/0 8 ==== osd_map(50..50 src has 1..50) v4 ==== 4077+0+0 (unknown 1200487953 0 0) 0x7f1c3c0895f0 con 0x7f1c4c0fb480 2024-09-11T08:00:32.596 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.592+0000 7f1c3affd640 1 -- 172.21.15.146:0/572573913 <== mon.1 v1:172.21.15.146:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f1c3c056b00 con 0x7f1c4c0fb480 2024-09-11T08:00:32.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:32 smithi146 ceph-mon[33158]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-11T08:00:32.759 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.759+0000 7f1c50fb5640 1 -- 172.21.15.146:0/572573913 --> v1:172.21.15.102:6800/2950771867 -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi102=foo.a", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c2c000bc0 con 0x7f1c3406f9a0 2024-09-11T08:00:32.766 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.766+0000 7f1c3affd640 1 -- 172.21.15.146:0/572573913 <== mgr.14150 v1:172.21.15.102:6800/2950771867 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (unknown 0 0 1123153589) 0x7f1c2c000bc0 con 0x7f1c3406f9a0 2024-09-11T08:00:32.766 INFO:teuthology.orchestra.run.smithi146.stdout:Scheduled rgw.foo.a update... 2024-09-11T08:00:32.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.767+0000 7f1c50fb5640 1 -- 172.21.15.146:0/572573913 >> v1:172.21.15.102:6800/2950771867 conn(0x7f1c3406f9a0 legacy=0x7f1c34071e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:32.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.767+0000 7f1c50fb5640 1 -- 172.21.15.146:0/572573913 >> v1:172.21.15.146:6789/0 conn(0x7f1c4c0fb480 legacy=0x7f1c4c065840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:32.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.767+0000 7f1c50fb5640 1 -- 172.21.15.146:0/572573913 shutdown_connections 2024-09-11T08:00:32.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.767+0000 7f1c50fb5640 1 -- 172.21.15.146:0/572573913 >> 172.21.15.146:0/572573913 conn(0x7f1c4c0f6c20 msgr2=0x7f1c4c1059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:00:32.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.768+0000 7f1c50fb5640 1 -- 172.21.15.146:0/572573913 shutdown_connections 2024-09-11T08:00:32.768 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:32.768+0000 7f1c50fb5640 1 -- 172.21.15.146:0/572573913 wait complete. 2024-09-11T08:00:32.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:32 smithi102 ceph-mon[29849]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-11T08:00:32.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:32 smithi102 ceph-mon[41371]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-11T08:00:33.372 DEBUG:teuthology.orchestra.run.smithi102:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@rgw.foo.a.service 2024-09-11T08:00:33.375 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi146 2024-09-11T08:00:33.375 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-11T08:00:33.617 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:00:34.066 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:33 smithi146 ceph-mon[33158]: from='client.24358 v1:172.21.15.146:0/572573913' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi102=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:00:34.067 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:33 smithi146 ceph-mon[33158]: Saving service rgw.foo.a spec with placement smithi102=foo.a;count:1 2024-09-11T08:00:34.067 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:33 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:34.067 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:33 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:00:34.067 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:33 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:34.067 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:33 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:00:34.067 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:33 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:34.067 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:33 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T08:00:34.067 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:33 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T08:00:34.067 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:33 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:34.067 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:33 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:34.067 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:33 smithi146 ceph-mon[33158]: Deploying daemon rgw.foo.a on smithi102 2024-09-11T08:00:34.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[29849]: from='client.24358 v1:172.21.15.146:0/572573913' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi102=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:00:34.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[29849]: Saving service rgw.foo.a spec with placement smithi102=foo.a;count:1 2024-09-11T08:00:34.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:34.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:00:34.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:34.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:00:34.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:34.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T08:00:34.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T08:00:34.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:34.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:34.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[29849]: Deploying daemon rgw.foo.a on smithi102 2024-09-11T08:00:34.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[41371]: from='client.24358 v1:172.21.15.146:0/572573913' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi102=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:00:34.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[41371]: Saving service rgw.foo.a spec with placement smithi102=foo.a;count:1 2024-09-11T08:00:34.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:34.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:00:34.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:34.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:00:34.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:34.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T08:00:34.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T08:00:34.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:34.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:34.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:33 smithi102 ceph-mon[41371]: Deploying daemon rgw.foo.a on smithi102 2024-09-11T08:00:34.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:34 smithi102 ceph-mon[29849]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-11T08:00:34.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:34 smithi102 ceph-mon[41371]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-11T08:00:34.897 INFO:journalctl@ceph.rgw.foo.a.smithi102.stdout:Sep 11 08:00:34 smithi102 podman[69519]: 2024-09-11 08:00:34.866368013 +0000 UTC m=+0.143072223 container create 58d5114f6458a81816dd1f418e5796a55dbd7776e9029633c300606a39dfab3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-11T08:00:34.982 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:34 smithi146 ceph-mon[33158]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-11T08:00:35.214 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.197+0000 7f044fade640 1 -- 172.21.15.146:0/3309134064 >> v1:172.21.15.146:6789/0 conn(0x7f04480ff980 legacy=0x7f0448101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:35.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.197+0000 7f044fade640 1 -- 172.21.15.146:0/3309134064 shutdown_connections 2024-09-11T08:00:35.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.197+0000 7f044fade640 1 -- 172.21.15.146:0/3309134064 >> 172.21.15.146:0/3309134064 conn(0x7f04480f6c20 msgr2=0x7f04480f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:00:35.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.197+0000 7f044fade640 1 -- 172.21.15.146:0/3309134064 shutdown_connections 2024-09-11T08:00:35.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.197+0000 7f044fade640 1 -- 172.21.15.146:0/3309134064 wait complete. 2024-09-11T08:00:35.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.198+0000 7f044fade640 1 Processor -- start 2024-09-11T08:00:35.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.198+0000 7f044fade640 1 -- start start 2024-09-11T08:00:35.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.198+0000 7f044fade640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0448117710 con 0x7f04480fb480 2024-09-11T08:00:35.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.198+0000 7f044fade640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0448118910 con 0x7f04481035a0 2024-09-11T08:00:35.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.198+0000 7f044fade640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0448119b10 con 0x7f04480ff980 2024-09-11T08:00:35.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.198+0000 7f044d079640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7f04480ff980 0x7f0448112460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.146:43410/0 (socket says 172.21.15.146:43410) 2024-09-11T08:00:35.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.198+0000 7f044e07b640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f04481035a0 0x7f0448115e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:59868/0 (socket says 172.21.15.146:59868) 2024-09-11T08:00:35.215 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.198+0000 7f044d079640 1 -- 172.21.15.146:0/3420637401 learned_addr learned my addr 172.21.15.146:0/3420637401 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-11T08:00:35.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.198+0000 7f044d87a640 1 --1- 172.21.15.146:0/3420637401 >> v1:172.21.15.102:6789/0 conn(0x7f04480fb480 0x7f0448065840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.146:48460/0 (socket says 172.21.15.146:48460) 2024-09-11T08:00:35.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.198+0000 7f043e7fc640 1 -- 172.21.15.146:0/3420637401 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 427588342 0 0) 0x7f0448118910 con 0x7f04481035a0 2024-09-11T08:00:35.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.198+0000 7f043e7fc640 1 -- 172.21.15.146:0/3420637401 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0434003720 con 0x7f04481035a0 2024-09-11T08:00:35.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.198+0000 7f043e7fc640 1 -- 172.21.15.146:0/3420637401 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896909867 0 0) 0x7f0448119b10 con 0x7f04480ff980 2024-09-11T08:00:35.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.198+0000 7f043e7fc640 1 -- 172.21.15.146:0/3420637401 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0448118910 con 0x7f04480ff980 2024-09-11T08:00:35.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.199+0000 7f043e7fc640 1 -- 172.21.15.146:0/3420637401 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3068158110 0 0) 0x7f0448118910 con 0x7f04480ff980 2024-09-11T08:00:35.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.199+0000 7f043e7fc640 1 -- 172.21.15.146:0/3420637401 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0448119b10 con 0x7f04480ff980 2024-09-11T08:00:35.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.199+0000 7f043e7fc640 1 -- 172.21.15.146:0/3420637401 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f0444002d50 con 0x7f04480ff980 2024-09-11T08:00:35.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.199+0000 7f043e7fc640 1 -- 172.21.15.146:0/3420637401 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2121573482 0 0) 0x7f0448119b10 con 0x7f04480ff980 2024-09-11T08:00:35.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.199+0000 7f043e7fc640 1 -- 172.21.15.146:0/3420637401 >> v1:172.21.15.146:6789/0 conn(0x7f04481035a0 legacy=0x7f0448115e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:35.216 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.199+0000 7f043e7fc640 1 -- 172.21.15.146:0/3420637401 >> v1:172.21.15.102:6789/0 conn(0x7f04480fb480 legacy=0x7f0448065840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:35.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.199+0000 7f043e7fc640 1 -- 172.21.15.146:0/3420637401 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044811ad10 con 0x7f04480ff980 2024-09-11T08:00:35.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.199+0000 7f044fade640 1 -- 172.21.15.146:0/3420637401 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0448118b00 con 0x7f04480ff980 2024-09-11T08:00:35.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.199+0000 7f044fade640 1 -- 172.21.15.146:0/3420637401 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0448118fd0 con 0x7f04480ff980 2024-09-11T08:00:35.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.199+0000 7f043e7fc640 1 -- 172.21.15.146:0/3420637401 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f04440024a0 con 0x7f04480ff980 2024-09-11T08:00:35.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.199+0000 7f043e7fc640 1 -- 172.21.15.146:0/3420637401 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f0444003800 con 0x7f04480ff980 2024-09-11T08:00:35.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.200+0000 7f044fade640 1 -- 172.21.15.146:0/3420637401 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f042c005040 con 0x7f04480ff980 2024-09-11T08:00:35.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.200+0000 7f043e7fc640 1 -- 172.21.15.146:0/3420637401 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 1029906827 0 0) 0x7f0444002610 con 0x7f04480ff980 2024-09-11T08:00:35.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.201+0000 7f043e7fc640 1 -- 172.21.15.146:0/3420637401 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(50..50 src has 1..50) v4 ==== 4077+0+0 (unknown 1200487953 0 0) 0x7f0444089ab0 con 0x7f04480ff980 2024-09-11T08:00:35.217 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.205+0000 7f043e7fc640 1 -- 172.21.15.146:0/3420637401 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f0444056fc0 con 0x7f04480ff980 2024-09-11T08:00:35.364 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.363+0000 7f044fade640 1 -- 172.21.15.146:0/3420637401 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) v1 -- 0x7f042c005870 con 0x7f04480ff980 2024-09-11T08:00:35.395 INFO:journalctl@ceph.rgw.foo.a.smithi102.stdout:Sep 11 08:00:35 smithi102 podman[69519]: 2024-09-11 08:00:35.093933246 +0000 UTC m=+0.370637452 container init 58d5114f6458a81816dd1f418e5796a55dbd7776e9029633c300606a39dfab3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-rgw-foo-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T08:00:35.396 INFO:journalctl@ceph.rgw.foo.a.smithi102.stdout:Sep 11 08:00:35 smithi102 podman[69519]: 2024-09-11 08:00:35.09865108 +0000 UTC m=+0.375355287 container start 58d5114f6458a81816dd1f418e5796a55dbd7776e9029633c300606a39dfab3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-rgw-foo-a, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-11T08:00:35.396 INFO:journalctl@ceph.rgw.foo.a.smithi102.stdout:Sep 11 08:00:35 smithi102 bash[69519]: 58d5114f6458a81816dd1f418e5796a55dbd7776e9029633c300606a39dfab3a 2024-09-11T08:00:35.396 INFO:journalctl@ceph.rgw.foo.a.smithi102.stdout:Sep 11 08:00:35 smithi102 systemd[1]: Started Ceph rgw.foo.a for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T08:00:35.792 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.791+0000 7f043e7fc640 1 -- 172.21.15.146:0/3420637401 <== mon.2 v1:172.21.15.102:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'datapool' created v51) v1 ==== 160+0+0 (unknown 2978880215 0 0) 0x7f0444056960 con 0x7f04480ff980 2024-09-11T08:00:35.792 INFO:teuthology.orchestra.run.smithi146.stderr:pool 'datapool' created 2024-09-11T08:00:35.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.795+0000 7f044fade640 1 -- 172.21.15.146:0/3420637401 >> v1:172.21.15.102:6800/2950771867 conn(0x7f043406fa40 legacy=0x7f0434071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:35.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.795+0000 7f044fade640 1 -- 172.21.15.146:0/3420637401 >> v1:172.21.15.102:6790/0 conn(0x7f04480ff980 legacy=0x7f0448112460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:35.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.795+0000 7f044fade640 1 -- 172.21.15.146:0/3420637401 shutdown_connections 2024-09-11T08:00:35.796 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.795+0000 7f044fade640 1 -- 172.21.15.146:0/3420637401 >> 172.21.15.146:0/3420637401 conn(0x7f04480f6c20 msgr2=0x7f0448105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:00:35.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.796+0000 7f044fade640 1 -- 172.21.15.146:0/3420637401 shutdown_connections 2024-09-11T08:00:35.797 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:35.796+0000 7f044fade640 1 -- 172.21.15.146:0/3420637401 wait complete. 2024-09-11T08:00:36.126 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.146:0/3420637401' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:00:36.127 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:35 smithi146 ceph-mon[33158]: from='client.24356 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:00:36.127 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:35 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:36.127 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:35 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:36.127 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:35 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:36.127 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:35 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:36.127 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:35 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:36.127 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:35 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:00:36.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.146:0/3420637401' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:00:36.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:35 smithi102 ceph-mon[29849]: from='client.24356 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:00:36.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:35 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:36.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:35 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:36.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:35 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:36.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:35 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:36.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:35 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:36.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:35 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:00:36.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.146:0/3420637401' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:00:36.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:35 smithi102 ceph-mon[41371]: from='client.24356 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T08:00:36.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:35 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:36.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:35 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:36.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:35 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:36.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:35 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:36.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:35 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:36.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:35 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:00:36.426 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- rbd pool init datapool 2024-09-11T08:00:36.668 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:00:37.088 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:36 smithi146 ceph-mon[33158]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:00:37.089 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:36 smithi146 ceph-mon[33158]: Saving service rgw.foo.a spec with placement smithi102=foo.a;count:1 2024-09-11T08:00:37.089 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:36 smithi146 ceph-mon[33158]: from='client.24356 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T08:00:37.089 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:36 smithi146 ceph-mon[33158]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T08:00:37.089 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:36 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4067860940' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:00:37.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:36 smithi102 ceph-mon[29849]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:00:37.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:36 smithi102 ceph-mon[29849]: Saving service rgw.foo.a spec with placement smithi102=foo.a;count:1 2024-09-11T08:00:37.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:36 smithi102 ceph-mon[29849]: from='client.24356 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T08:00:37.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:36 smithi102 ceph-mon[29849]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T08:00:37.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:36 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4067860940' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:00:37.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:36 smithi102 ceph-mon[41371]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T08:00:37.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:36 smithi102 ceph-mon[41371]: Saving service rgw.foo.a spec with placement smithi102=foo.a;count:1 2024-09-11T08:00:37.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:36 smithi102 ceph-mon[41371]: from='client.24356 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T08:00:37.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:36 smithi102 ceph-mon[41371]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T08:00:37.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:36 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4067860940' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T08:00:38.072 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4067860940' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T08:00:38.073 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:37 smithi146 ceph-mon[33158]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T08:00:38.073 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:37 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:38.073 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:37 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:38.073 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:37 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:38.073 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:37 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:00:38.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4067860940' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T08:00:38.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:37 smithi102 ceph-mon[29849]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T08:00:38.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:37 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:38.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:37 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:38.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:37 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:38.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:37 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:00:38.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4067860940' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T08:00:38.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:37 smithi102 ceph-mon[41371]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T08:00:38.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:37 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:38.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:37 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:38.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:37 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:38.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:37 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:00:39.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[41371]: Checking dashboard <-> RGW credentials 2024-09-11T08:00:39.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[41371]: pgmap v169: 36 pgs: 7 creating+peering, 28 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:00:39.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[41371]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T08:00:39.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4067860940' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:00:39.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1849326121' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:00:39.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[41371]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:00:39.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.146:0/1168341450' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:00:39.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[41371]: from='client.24397 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:00:39.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:39.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4067860940' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:00:39.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1849326121' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:00:39.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[41371]: from='client.24397 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T08:00:39.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[41371]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T08:00:39.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[29849]: Checking dashboard <-> RGW credentials 2024-09-11T08:00:39.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[29849]: pgmap v169: 36 pgs: 7 creating+peering, 28 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:00:39.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[29849]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T08:00:39.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4067860940' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:00:39.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1849326121' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:00:39.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[29849]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:00:39.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.146:0/1168341450' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:00:39.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[29849]: from='client.24397 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:00:39.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:39.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4067860940' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:00:39.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1849326121' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:00:39.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[29849]: from='client.24397 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T08:00:39.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:38 smithi102 ceph-mon[29849]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T08:00:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:38 smithi146 ceph-mon[33158]: Checking dashboard <-> RGW credentials 2024-09-11T08:00:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:38 smithi146 ceph-mon[33158]: pgmap v169: 36 pgs: 7 creating+peering, 28 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T08:00:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:38 smithi146 ceph-mon[33158]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T08:00:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4067860940' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:00:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1849326121' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T08:00:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:38 smithi146 ceph-mon[33158]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:00:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.146:0/1168341450' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:00:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:38 smithi146 ceph-mon[33158]: from='client.24397 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T08:00:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:38 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:39.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4067860940' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:00:39.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1849326121' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T08:00:39.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:38 smithi146 ceph-mon[33158]: from='client.24397 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T08:00:39.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:38 smithi146 ceph-mon[33158]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T08:00:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:39 smithi102 ceph-mon[29849]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:00:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:39 smithi102 ceph-mon[29849]: Cluster is now healthy 2024-09-11T08:00:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:39 smithi102 ceph-mon[29849]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T08:00:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:39 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1849326121' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:00:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:39 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4067860940' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:00:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:39 smithi102 ceph-mon[41371]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:00:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:39 smithi102 ceph-mon[41371]: Cluster is now healthy 2024-09-11T08:00:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:39 smithi102 ceph-mon[41371]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T08:00:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:39 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1849326121' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:00:40.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:39 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4067860940' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:00:40.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:39 smithi146 ceph-mon[33158]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T08:00:40.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:39 smithi146 ceph-mon[33158]: Cluster is now healthy 2024-09-11T08:00:40.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:39 smithi146 ceph-mon[33158]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T08:00:40.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:39 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1849326121' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:00:40.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:39 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4067860940' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T08:00:41.139 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:40 smithi146 ceph-mon[33158]: pgmap v172: 68 pgs: 30 creating+peering, 24 unknown, 14 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:00:41.140 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1849326121' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:00:41.140 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4067860940' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:00:41.140 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:40 smithi146 ceph-mon[33158]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T08:00:41.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:40 smithi102 ceph-mon[29849]: pgmap v172: 68 pgs: 30 creating+peering, 24 unknown, 14 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:00:41.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1849326121' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:00:41.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4067860940' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:00:41.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:40 smithi102 ceph-mon[29849]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T08:00:41.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:40 smithi102 ceph-mon[41371]: pgmap v172: 68 pgs: 30 creating+peering, 24 unknown, 14 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:00:41.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1849326121' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:00:41.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4067860940' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T08:00:41.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:40 smithi102 ceph-mon[41371]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T08:00:41.445 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.146 --placement '1;smithi146=iscsi.a' 2024-09-11T08:00:41.694 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:00:42.862 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:42 smithi146 ceph-mon[33158]: pgmap v175: 100 pgs: 31 creating+peering, 20 unknown, 49 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-11T08:00:42.862 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:42 smithi146 ceph-mon[33158]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T08:00:42.862 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/502152468' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:00:42.862 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3739207953' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:00:42.862 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:42 smithi146 ceph-mon[33158]: from='client.24395 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:00:42.862 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:42 smithi146 ceph-mon[33158]: from='client.24398 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:00:43.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:42 smithi102 ceph-mon[29849]: pgmap v175: 100 pgs: 31 creating+peering, 20 unknown, 49 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-11T08:00:43.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:42 smithi102 ceph-mon[29849]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T08:00:43.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/502152468' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:00:43.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3739207953' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:00:43.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:42 smithi102 ceph-mon[29849]: from='client.24395 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:00:43.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:42 smithi102 ceph-mon[29849]: from='client.24398 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:00:43.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:42 smithi102 ceph-mon[41371]: pgmap v175: 100 pgs: 31 creating+peering, 20 unknown, 49 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-11T08:00:43.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:42 smithi102 ceph-mon[41371]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T08:00:43.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/502152468' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:00:43.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3739207953' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:00:43.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:42 smithi102 ceph-mon[41371]: from='client.24395 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:00:43.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:42 smithi102 ceph-mon[41371]: from='client.24398 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T08:00:43.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.383+0000 7f7258c0a640 1 -- 172.21.15.146:0/1617364405 >> v1:172.21.15.146:6789/0 conn(0x7f72540fb470 legacy=0x7f72540fb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:43.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.384+0000 7f7258c0a640 1 -- 172.21.15.146:0/1617364405 shutdown_connections 2024-09-11T08:00:43.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.384+0000 7f7258c0a640 1 -- 172.21.15.146:0/1617364405 >> 172.21.15.146:0/1617364405 conn(0x7f72540f6c30 msgr2=0x7f72540f9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:00:43.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.384+0000 7f7258c0a640 1 -- 172.21.15.146:0/1617364405 shutdown_connections 2024-09-11T08:00:43.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.384+0000 7f7258c0a640 1 -- 172.21.15.146:0/1617364405 wait complete. 2024-09-11T08:00:43.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.384+0000 7f7258c0a640 1 Processor -- start 2024-09-11T08:00:43.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.384+0000 7f7258c0a640 1 -- start start 2024-09-11T08:00:43.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.384+0000 7f7258c0a640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f725410d270 con 0x7f72540fb470 2024-09-11T08:00:43.399 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.384+0000 7f7258c0a640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7254118a70 con 0x7f72540ff970 2024-09-11T08:00:43.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.384+0000 7f7258c0a640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7254119c10 con 0x7f7254103590 2024-09-11T08:00:43.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.384+0000 7f7251d9b640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f72540ff970 0x7f7254112c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:59916/0 (socket says 172.21.15.146:59916) 2024-09-11T08:00:43.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.384+0000 7f7251d9b640 1 -- 172.21.15.146:0/1034841211 learned_addr learned my addr 172.21.15.146:0/1034841211 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-11T08:00:43.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.385+0000 7f7242ffd640 1 -- 172.21.15.146:0/1034841211 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2637329023 0 0) 0x7f7254118a70 con 0x7f72540ff970 2024-09-11T08:00:43.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.385+0000 7f7242ffd640 1 -- 172.21.15.146:0/1034841211 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f723c0034e0 con 0x7f72540ff970 2024-09-11T08:00:43.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.385+0000 7f7242ffd640 1 -- 172.21.15.146:0/1034841211 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208133681 0 0) 0x7f725410d270 con 0x7f72540fb470 2024-09-11T08:00:43.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.385+0000 7f7242ffd640 1 -- 172.21.15.146:0/1034841211 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7254118a70 con 0x7f72540fb470 2024-09-11T08:00:43.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.385+0000 7f7242ffd640 1 -- 172.21.15.146:0/1034841211 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577171005 0 0) 0x7f7254119c10 con 0x7f7254103590 2024-09-11T08:00:43.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.385+0000 7f7242ffd640 1 -- 172.21.15.146:0/1034841211 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f725410d270 con 0x7f7254103590 2024-09-11T08:00:43.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.385+0000 7f7242ffd640 1 -- 172.21.15.146:0/1034841211 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4148789582 0 0) 0x7f723c0034e0 con 0x7f72540ff970 2024-09-11T08:00:43.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.385+0000 7f7242ffd640 1 -- 172.21.15.146:0/1034841211 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7254119c10 con 0x7f72540ff970 2024-09-11T08:00:43.400 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.385+0000 7f7242ffd640 1 -- 172.21.15.146:0/1034841211 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f7248001940 con 0x7f72540ff970 2024-09-11T08:00:43.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.385+0000 7f7242ffd640 1 -- 172.21.15.146:0/1034841211 <== mon.1 v1:172.21.15.146:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2740384720 0 0) 0x7f7254119c10 con 0x7f72540ff970 2024-09-11T08:00:43.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.385+0000 7f7242ffd640 1 -- 172.21.15.146:0/1034841211 >> v1:172.21.15.102:6790/0 conn(0x7f7254103590 legacy=0x7f7254116340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:43.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.385+0000 7f7242ffd640 1 -- 172.21.15.146:0/1034841211 >> v1:172.21.15.102:6789/0 conn(0x7f72540fb470 legacy=0x7f725410c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:43.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.385+0000 7f7242ffd640 1 -- 172.21.15.146:0/1034841211 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725411adf0 con 0x7f72540ff970 2024-09-11T08:00:43.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.385+0000 7f7258c0a640 1 -- 172.21.15.146:0/1034841211 --> v1:172.21.15.146:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7254117ac0 con 0x7f72540ff970 2024-09-11T08:00:43.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.385+0000 7f7258c0a640 1 -- 172.21.15.146:0/1034841211 --> v1:172.21.15.146:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7254117f10 con 0x7f72540ff970 2024-09-11T08:00:43.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.385+0000 7f7242ffd640 1 -- 172.21.15.146:0/1034841211 <== mon.1 v1:172.21.15.146:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f7248002410 con 0x7f72540ff970 2024-09-11T08:00:43.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.385+0000 7f7242ffd640 1 -- 172.21.15.146:0/1034841211 <== mon.1 v1:172.21.15.146:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f7248005a80 con 0x7f72540ff970 2024-09-11T08:00:43.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.386+0000 7f7258c0a640 1 -- 172.21.15.146:0/1034841211 --> v1:172.21.15.146:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7234005040 con 0x7f72540ff970 2024-09-11T08:00:43.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.387+0000 7f7242ffd640 1 -- 172.21.15.146:0/1034841211 <== mon.1 v1:172.21.15.146:6789/0 7 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 1029906827 0 0) 0x7f7248001fc0 con 0x7f72540ff970 2024-09-11T08:00:43.401 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.387+0000 7f7242ffd640 1 -- 172.21.15.146:0/1034841211 <== mon.1 v1:172.21.15.146:6789/0 8 ==== osd_map(58..58 src has 1..58) v4 ==== 5922+0+0 (unknown 594621595 0 0) 0x7f7248089cf0 con 0x7f72540ff970 2024-09-11T08:00:43.402 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.392+0000 7f7242ffd640 1 -- 172.21.15.146:0/1034841211 <== mon.1 v1:172.21.15.146:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f7248056b30 con 0x7f72540ff970 2024-09-11T08:00:43.556 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.556+0000 7f7258c0a640 1 -- 172.21.15.146:0/1034841211 --> v1:172.21.15.102:6800/2950771867 -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.146", "placement": "1;smithi146=iscsi.a", "target": ["mon-mgr", ""]}) v1 -- 0x7f7234000c90 con 0x7f723c06fad0 2024-09-11T08:00:43.564 INFO:teuthology.orchestra.run.smithi146.stdout:Scheduled iscsi.datapool update... 2024-09-11T08:00:43.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.563+0000 7f7242ffd640 1 -- 172.21.15.146:0/1034841211 <== mgr.14150 v1:172.21.15.102:6800/2950771867 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+35 (unknown 0 0 803663096) 0x7f7234000c90 con 0x7f723c06fad0 2024-09-11T08:00:43.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.565+0000 7f7258c0a640 1 -- 172.21.15.146:0/1034841211 >> v1:172.21.15.102:6800/2950771867 conn(0x7f723c06fad0 legacy=0x7f723c071f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:43.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.565+0000 7f7258c0a640 1 -- 172.21.15.146:0/1034841211 >> v1:172.21.15.146:6789/0 conn(0x7f72540ff970 legacy=0x7f7254112c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:43.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.565+0000 7f7258c0a640 1 -- 172.21.15.146:0/1034841211 shutdown_connections 2024-09-11T08:00:43.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.565+0000 7f7258c0a640 1 -- 172.21.15.146:0/1034841211 >> 172.21.15.146:0/1034841211 conn(0x7f72540f6c30 msgr2=0x7f72540f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:00:43.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.565+0000 7f7258c0a640 1 -- 172.21.15.146:0/1034841211 shutdown_connections 2024-09-11T08:00:43.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:43.566+0000 7f7258c0a640 1 -- 172.21.15.146:0/1034841211 wait complete. 2024-09-11T08:00:44.137 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:43 smithi146 ceph-mon[33158]: from='client.24395 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:00:44.137 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:43 smithi146 ceph-mon[33158]: from='client.24398 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:00:44.137 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:43 smithi146 ceph-mon[33158]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T08:00:44.137 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:43 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3739207953' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:00:44.137 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:43 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/502152468' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:00:44.137 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:43 smithi146 ceph-mon[33158]: from='client.24398 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:00:44.137 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:43 smithi146 ceph-mon[33158]: from='client.24395 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:00:44.137 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:43 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:44.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:43 smithi102 ceph-mon[29849]: from='client.24395 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:00:44.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:43 smithi102 ceph-mon[29849]: from='client.24398 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:00:44.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:43 smithi102 ceph-mon[29849]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T08:00:44.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:43 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3739207953' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:00:44.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:43 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/502152468' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:00:44.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:43 smithi102 ceph-mon[29849]: from='client.24398 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:00:44.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:43 smithi102 ceph-mon[29849]: from='client.24395 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:00:44.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:43 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:44.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:43 smithi102 ceph-mon[41371]: from='client.24395 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:00:44.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:43 smithi102 ceph-mon[41371]: from='client.24398 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T08:00:44.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:43 smithi102 ceph-mon[41371]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T08:00:44.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:43 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3739207953' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:00:44.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:43 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/502152468' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:00:44.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:43 smithi102 ceph-mon[41371]: from='client.24398 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:00:44.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:43 smithi102 ceph-mon[41371]: from='client.24395 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T08:00:44.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:43 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:44.435 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-11T08:00:44.436 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-11T08:00:44.436 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-11T08:00:44.464 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-11T08:00:44.464 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-11T08:00:44.491 DEBUG:teuthology.orchestra.run.smithi146:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@iscsi.iscsi.a.service 2024-09-11T08:00:44.534 INFO:tasks.cephadm:Adding prometheus.a on smithi146 2024-09-11T08:00:44.534 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph orch apply prometheus '1;smithi146=a' 2024-09-11T08:00:44.922 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:44 smithi146 ceph-mon[33158]: pgmap v178: 132 pgs: 1 creating+activating, 24 creating+peering, 35 unknown, 72 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-11T08:00:44.923 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:44 smithi146 ceph-mon[33158]: from='client.24415 v1:172.21.15.146:0/1034841211' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.146", "placement": "1;smithi146=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:00:44.923 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:44 smithi146 ceph-mon[33158]: Saving service iscsi.datapool spec with placement smithi146=iscsi.a;count:1 2024-09-11T08:00:44.923 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:44 smithi146 ceph-mon[33158]: from='client.24398 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:00:44.923 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:44 smithi146 ceph-mon[33158]: from='client.24395 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:00:44.923 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:44 smithi146 ceph-mon[33158]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T08:00:44.923 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:44 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:44.923 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:44 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:44.923 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:44 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:44.923 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:44 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T08:00:44.923 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:44 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T08:00:44.923 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:44 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:45.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[29849]: pgmap v178: 132 pgs: 1 creating+activating, 24 creating+peering, 35 unknown, 72 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-11T08:00:45.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[29849]: from='client.24415 v1:172.21.15.146:0/1034841211' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.146", "placement": "1;smithi146=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:00:45.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[29849]: Saving service iscsi.datapool spec with placement smithi146=iscsi.a;count:1 2024-09-11T08:00:45.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[29849]: from='client.24398 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:00:45.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[29849]: from='client.24395 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:00:45.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[29849]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T08:00:45.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:45.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:45.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:45.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T08:00:45.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T08:00:45.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:45.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[41371]: pgmap v178: 132 pgs: 1 creating+activating, 24 creating+peering, 35 unknown, 72 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-11T08:00:45.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[41371]: from='client.24415 v1:172.21.15.146:0/1034841211' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.146", "placement": "1;smithi146=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:00:45.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[41371]: Saving service iscsi.datapool spec with placement smithi146=iscsi.a;count:1 2024-09-11T08:00:45.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[41371]: from='client.24398 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:00:45.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[41371]: from='client.24395 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T08:00:45.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[41371]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T08:00:45.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:45.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:45.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:45.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T08:00:45.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T08:00:45.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:44 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:45.250 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:00:46.039 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:45 smithi146 ceph-mon[33158]: Deploying daemon iscsi.iscsi.a on smithi146 2024-09-11T08:00:46.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:45 smithi102 ceph-mon[41371]: Deploying daemon iscsi.iscsi.a on smithi146 2024-09-11T08:00:46.149 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:45 smithi102 ceph-mon[29849]: Deploying daemon iscsi.iscsi.a on smithi146 2024-09-11T08:00:46.326 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:46 smithi146 systemd[1]: Starting Ceph iscsi.iscsi.a for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:00:47.091 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:46 smithi146 ceph-mon[33158]: pgmap v180: 132 pgs: 1 creating+activating, 3 unknown, 128 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 1.7 KiB/s wr, 38 op/s 2024-09-11T08:00:47.091 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:46 smithi146 ceph-mon[33158]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T08:00:47.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:46 smithi102 ceph-mon[29849]: pgmap v180: 132 pgs: 1 creating+activating, 3 unknown, 128 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 1.7 KiB/s wr, 38 op/s 2024-09-11T08:00:47.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:46 smithi102 ceph-mon[29849]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T08:00:47.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:46 smithi102 ceph-mon[41371]: pgmap v180: 132 pgs: 1 creating+activating, 3 unknown, 128 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 1.7 KiB/s wr, 38 op/s 2024-09-11T08:00:47.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:46 smithi102 ceph-mon[41371]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T08:00:47.342 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:47 smithi146 podman[62533]: 2024-09-11 08:00:47.110767206 +0000 UTC m=+0.247788024 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:00:47.342 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:47 smithi146 podman[62533]: 2024-09-11 08:00:47.224992642 +0000 UTC m=+0.362013458 container create 810393cb3f2c802c90e3ccca04d945f729f8bdd99467fb28929375f5408c493d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-11T08:00:47.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.646+0000 7f418cba5640 1 -- 172.21.15.146:0/3484698821 >> v1:172.21.15.146:6789/0 conn(0x7f4188100a90 legacy=0x7f4188102e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:47.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.646+0000 7f418cba5640 1 -- 172.21.15.146:0/3484698821 shutdown_connections 2024-09-11T08:00:47.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.646+0000 7f418cba5640 1 -- 172.21.15.146:0/3484698821 >> 172.21.15.146:0/3484698821 conn(0x7f41880f7d30 msgr2=0x7f41880fa180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:00:47.665 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.646+0000 7f418cba5640 1 -- 172.21.15.146:0/3484698821 shutdown_connections 2024-09-11T08:00:47.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.646+0000 7f418cba5640 1 -- 172.21.15.146:0/3484698821 wait complete. 2024-09-11T08:00:47.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.647+0000 7f418cba5640 1 Processor -- start 2024-09-11T08:00:47.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.647+0000 7f418cba5640 1 -- start start 2024-09-11T08:00:47.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.647+0000 7f418cba5640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f418810d1e0 con 0x7f41881046b0 2024-09-11T08:00:47.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.647+0000 7f418cba5640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f41881189e0 con 0x7f41880fc590 2024-09-11T08:00:47.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.647+0000 7f418cba5640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4188119b80 con 0x7f4188100a90 2024-09-11T08:00:47.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.647+0000 7f418659c640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f41880fc590 0x7f418810c650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:57868/0 (socket says 172.21.15.146:57868) 2024-09-11T08:00:47.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.647+0000 7f418659c640 1 -- 172.21.15.146:0/273893184 learned_addr learned my addr 172.21.15.146:0/273893184 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-11T08:00:47.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.647+0000 7f4176ffd640 1 -- 172.21.15.146:0/273893184 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1945074335 0 0) 0x7f41881189e0 con 0x7f41880fc590 2024-09-11T08:00:47.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.647+0000 7f4176ffd640 1 -- 172.21.15.146:0/273893184 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4170003120 con 0x7f41880fc590 2024-09-11T08:00:47.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.647+0000 7f4176ffd640 1 -- 172.21.15.146:0/273893184 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1814028671 0 0) 0x7f4188119b80 con 0x7f4188100a90 2024-09-11T08:00:47.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.647+0000 7f4176ffd640 1 -- 172.21.15.146:0/273893184 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41881189e0 con 0x7f4188100a90 2024-09-11T08:00:47.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.647+0000 7f4176ffd640 1 -- 172.21.15.146:0/273893184 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 682756822 0 0) 0x7f418810d1e0 con 0x7f41881046b0 2024-09-11T08:00:47.666 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.647+0000 7f4176ffd640 1 -- 172.21.15.146:0/273893184 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4188119b80 con 0x7f41881046b0 2024-09-11T08:00:47.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.647+0000 7f4176ffd640 1 -- 172.21.15.146:0/273893184 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1332759241 0 0) 0x7f4170003120 con 0x7f41880fc590 2024-09-11T08:00:47.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.647+0000 7f4176ffd640 1 -- 172.21.15.146:0/273893184 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f418810d1e0 con 0x7f41880fc590 2024-09-11T08:00:47.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.647+0000 7f4176ffd640 1 -- 172.21.15.146:0/273893184 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f41780018b0 con 0x7f41880fc590 2024-09-11T08:00:47.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.648+0000 7f4176ffd640 1 -- 172.21.15.146:0/273893184 <== mon.1 v1:172.21.15.146:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3048213320 0 0) 0x7f418810d1e0 con 0x7f41880fc590 2024-09-11T08:00:47.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.648+0000 7f4176ffd640 1 -- 172.21.15.146:0/273893184 >> v1:172.21.15.102:6790/0 conn(0x7f4188100a90 legacy=0x7f4188112b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:47.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.648+0000 7f4176ffd640 1 -- 172.21.15.146:0/273893184 >> v1:172.21.15.102:6789/0 conn(0x7f41881046b0 legacy=0x7f41881162b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:47.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.648+0000 7f4176ffd640 1 -- 172.21.15.146:0/273893184 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418811ad60 con 0x7f41880fc590 2024-09-11T08:00:47.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.648+0000 7f418cba5640 1 -- 172.21.15.146:0/273893184 --> v1:172.21.15.146:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4188117a30 con 0x7f41880fc590 2024-09-11T08:00:47.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.648+0000 7f418cba5640 1 -- 172.21.15.146:0/273893184 --> v1:172.21.15.146:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4188117e80 con 0x7f41880fc590 2024-09-11T08:00:47.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.648+0000 7f4176ffd640 1 -- 172.21.15.146:0/273893184 <== mon.1 v1:172.21.15.146:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f4178002350 con 0x7f41880fc590 2024-09-11T08:00:47.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.648+0000 7f4176ffd640 1 -- 172.21.15.146:0/273893184 <== mon.1 v1:172.21.15.146:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f4178005a10 con 0x7f41880fc590 2024-09-11T08:00:47.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.649+0000 7f418cba5640 1 -- 172.21.15.146:0/273893184 --> v1:172.21.15.146:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4168005040 con 0x7f41880fc590 2024-09-11T08:00:47.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.649+0000 7f4176ffd640 1 -- 172.21.15.146:0/273893184 <== mon.1 v1:172.21.15.146:6789/0 7 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 1029906827 0 0) 0x7f4178001f00 con 0x7f41880fc590 2024-09-11T08:00:47.667 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.650+0000 7f4176ffd640 1 -- 172.21.15.146:0/273893184 <== mon.1 v1:172.21.15.146:6789/0 8 ==== osd_map(60..60 src has 1..60) v4 ==== 5938+0+0 (unknown 1150950379 0 0) 0x7f4178089cb0 con 0x7f41880fc590 2024-09-11T08:00:47.668 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.654+0000 7f4176ffd640 1 -- 172.21.15.146:0/273893184 <== mon.1 v1:172.21.15.146:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f4178056a80 con 0x7f41880fc590 2024-09-11T08:00:47.822 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.820+0000 7f418cba5640 1 -- 172.21.15.146:0/273893184 --> v1:172.21.15.102:6800/2950771867 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi146=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f4168000bc0 con 0x7f417006fc30 2024-09-11T08:00:47.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:47 smithi146 podman[62533]: 2024-09-11 08:00:47.447720916 +0000 UTC m=+0.584741735 container init 810393cb3f2c802c90e3ccca04d945f729f8bdd99467fb28929375f5408c493d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-11T08:00:47.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:47 smithi146 podman[62533]: 2024-09-11 08:00:47.45280006 +0000 UTC m=+0.589820885 container start 810393cb3f2c802c90e3ccca04d945f729f8bdd99467fb28929375f5408c493d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T08:00:47.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:47 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu[62591]: log file path now is '/var/log/tcmu-runner.log' 2024-09-11T08:00:47.828 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.827+0000 7f4176ffd640 1 -- 172.21.15.146:0/273893184 <== mgr.14150 v1:172.21.15.102:6800/2950771867 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (unknown 0 0 1342662408) 0x7f4168000bc0 con 0x7f417006fc30 2024-09-11T08:00:47.828 INFO:teuthology.orchestra.run.smithi146.stdout:Scheduled prometheus update... 2024-09-11T08:00:47.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.829+0000 7f418cba5640 1 -- 172.21.15.146:0/273893184 >> v1:172.21.15.102:6800/2950771867 conn(0x7f417006fc30 legacy=0x7f41700720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:47.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.829+0000 7f418cba5640 1 -- 172.21.15.146:0/273893184 >> v1:172.21.15.146:6789/0 conn(0x7f41880fc590 legacy=0x7f418810c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:47.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.829+0000 7f418cba5640 1 -- 172.21.15.146:0/273893184 shutdown_connections 2024-09-11T08:00:47.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.829+0000 7f418cba5640 1 -- 172.21.15.146:0/273893184 >> 172.21.15.146:0/273893184 conn(0x7f41880f7d30 msgr2=0x7f4188106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:00:47.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.829+0000 7f418cba5640 1 -- 172.21.15.146:0/273893184 shutdown_connections 2024-09-11T08:00:47.830 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:47.829+0000 7f418cba5640 1 -- 172.21.15.146:0/273893184 wait complete. 2024-09-11T08:00:48.184 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:47 smithi146 bash[62533]: 810393cb3f2c802c90e3ccca04d945f729f8bdd99467fb28929375f5408c493d 2024-09-11T08:00:48.566 DEBUG:teuthology.orchestra.run.smithi146:prometheus.a> sudo journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@prometheus.a.service 2024-09-11T08:00:48.568 INFO:tasks.cephadm:Adding node-exporter.a on smithi102 2024-09-11T08:00:48.568 INFO:tasks.cephadm:Adding node-exporter.b on smithi146 2024-09-11T08:00:48.568 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph orch apply node-exporter '2;smithi102=a;smithi146=b' 2024-09-11T08:00:48.827 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:48 smithi146 podman[62734]: 2024-09-11 08:00:48.663693953 +0000 UTC m=+0.016633325 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:00:48.827 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:48 smithi146 podman[62734]: 2024-09-11 08:00:48.761700503 +0000 UTC m=+0.114639868 container create b1dab48cbbb6fcc4cb24f1ad9c901746782d38974d9c0a21d4e0e4445b9c18b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-11T08:00:49.135 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:00:49.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:48 smithi102 ceph-mon[29849]: pgmap v182: 132 pgs: 1 creating+activating, 131 active+clean; 580 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 28 KiB/s rd, 2.7 KiB/s wr, 67 op/s 2024-09-11T08:00:49.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:48 smithi102 ceph-mon[29849]: from='client.24436 v1:172.21.15.146:0/273893184' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi146=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:00:49.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:48 smithi102 ceph-mon[29849]: Saving service prometheus spec with placement smithi146=a;count:1 2024-09-11T08:00:49.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:48 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:49.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:48 smithi102 ceph-mon[41371]: pgmap v182: 132 pgs: 1 creating+activating, 131 active+clean; 580 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 28 KiB/s rd, 2.7 KiB/s wr, 67 op/s 2024-09-11T08:00:49.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:48 smithi102 ceph-mon[41371]: from='client.24436 v1:172.21.15.146:0/273893184' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi146=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:00:49.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:48 smithi102 ceph-mon[41371]: Saving service prometheus spec with placement smithi146=a;count:1 2024-09-11T08:00:49.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:48 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:49.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:48 smithi146 ceph-mon[33158]: pgmap v182: 132 pgs: 1 creating+activating, 131 active+clean; 580 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 28 KiB/s rd, 2.7 KiB/s wr, 67 op/s 2024-09-11T08:00:49.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:48 smithi146 ceph-mon[33158]: from='client.24436 v1:172.21.15.146:0/273893184' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi146=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:00:49.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:48 smithi146 ceph-mon[33158]: Saving service prometheus spec with placement smithi146=a;count:1 2024-09-11T08:00:49.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:48 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:49.184 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:48 smithi146 podman[62734]: 2024-09-11 08:00:48.980829942 +0000 UTC m=+0.333769306 container init b1dab48cbbb6fcc4cb24f1ad9c901746782d38974d9c0a21d4e0e4445b9c18b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:00:49.184 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:48 smithi146 podman[62734]: 2024-09-11 08:00:48.985310061 +0000 UTC m=+0.338249438 container start b1dab48cbbb6fcc4cb24f1ad9c901746782d38974d9c0a21d4e0e4445b9c18b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-11T08:00:49.184 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:49 smithi146 bash[62734]: b1dab48cbbb6fcc4cb24f1ad9c901746782d38974d9c0a21d4e0e4445b9c18b8 2024-09-11T08:00:49.184 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:49 smithi146 systemd[1]: Started Ceph iscsi.iscsi.a for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T08:00:49.567 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug Started the configuration object watcher 2024-09-11T08:00:49.567 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug Checking for config object changes every 1s 2024-09-11T08:00:49.567 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug Processing osd blocklist entries for this node 2024-09-11T08:00:49.567 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug Reading the configuration object to update local LIO configuration 2024-09-11T08:00:49.567 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug Configuration does not have an entry for this host(smithi146.front.sepia.ceph.com) - nothing to define to LIO 2024-09-11T08:00:49.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-11T08:00:49.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: * Environment: production 2024-09-11T08:00:49.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T08:00:49.935 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: Use a production WSGI server instead. 2024-09-11T08:00:49.935 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: * Debug mode: off 2024-09-11T08:00:49.935 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug * Running on all addresses. 2024-09-11T08:00:49.935 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T08:00:49.935 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: * Running on all addresses. 2024-09-11T08:00:49.935 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-11T08:00:49.935 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug * Running on http://[fe80::ec4:7aff:fe88:76fd]:5000/ (Press CTRL+C to quit) 2024-09-11T08:00:49.935 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: * Running on http://[fe80::ec4:7aff:fe88:76fd]:5000/ (Press CTRL+C to quit) 2024-09-11T08:00:50.313 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:50 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:50.313 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:50 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:50.313 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:50 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:50.313 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:50 smithi146 ceph-mon[33158]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T08:00:50.313 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:50 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:50.313 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:50 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:00:50.314 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:50 smithi146 ceph-mon[33158]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 70 KiB/s rd, 5.6 KiB/s wr, 169 op/s 2024-09-11T08:00:50.314 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:50 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.146:0/936945477' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T08:00:50.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:50 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:50.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:50 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:50.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:50 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:50.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:50 smithi102 ceph-mon[29849]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T08:00:50.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:50 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:50.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:50 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:00:50.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:50 smithi102 ceph-mon[29849]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 70 KiB/s rd, 5.6 KiB/s wr, 169 op/s 2024-09-11T08:00:50.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:50 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.146:0/936945477' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T08:00:50.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:50 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:50.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:50 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:50.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:50 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:50.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:50 smithi102 ceph-mon[41371]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T08:00:50.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:50 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:50.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:50 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:00:50.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:50 smithi102 ceph-mon[41371]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 70 KiB/s rd, 5.6 KiB/s wr, 169 op/s 2024-09-11T08:00:50.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:50 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.146:0/936945477' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T08:00:50.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.803+0000 7f08b7d86640 1 -- 172.21.15.146:0/353606621 >> v1:172.21.15.146:6789/0 conn(0x7f08b01046c0 legacy=0x7f08b0106b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:50.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.804+0000 7f08b7d86640 1 -- 172.21.15.146:0/353606621 shutdown_connections 2024-09-11T08:00:50.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.804+0000 7f08b7d86640 1 -- 172.21.15.146:0/353606621 >> 172.21.15.146:0/353606621 conn(0x7f08b00f63c0 msgr2=0x7f08b00f87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:00:50.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.804+0000 7f08b7d86640 1 -- 172.21.15.146:0/353606621 shutdown_connections 2024-09-11T08:00:50.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.804+0000 7f08b7d86640 1 -- 172.21.15.146:0/353606621 wait complete. 2024-09-11T08:00:50.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.804+0000 7f08b7d86640 1 Processor -- start 2024-09-11T08:00:50.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.804+0000 7f08b7d86640 1 -- start start 2024-09-11T08:00:50.817 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.804+0000 7f08b7d86640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f08b01197b0 con 0x7f08b01046c0 2024-09-11T08:00:50.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.804+0000 7f08b7d86640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f08b011a9b0 con 0x7f08b00fdbc0 2024-09-11T08:00:50.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.804+0000 7f08b7d86640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f08b011bbb0 con 0x7f08b0100bc0 2024-09-11T08:00:50.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.804+0000 7f08b5b22640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f08b00fdbc0 0x7f08b00fd160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:57956/0 (socket says 172.21.15.146:57956) 2024-09-11T08:00:50.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.804+0000 7f08b5b22640 1 -- 172.21.15.146:0/2734067889 learned_addr learned my addr 172.21.15.146:0/2734067889 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-11T08:00:50.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.805+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957224641 0 0) 0x7f08b011a9b0 con 0x7f08b00fdbc0 2024-09-11T08:00:50.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.805+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f089c003720 con 0x7f08b00fdbc0 2024-09-11T08:00:50.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.805+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2959640173 0 0) 0x7f08b011bbb0 con 0x7f08b0100bc0 2024-09-11T08:00:50.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.805+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08b011a9b0 con 0x7f08b0100bc0 2024-09-11T08:00:50.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.805+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3125955925 0 0) 0x7f08b01197b0 con 0x7f08b01046c0 2024-09-11T08:00:50.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.805+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08b011bbb0 con 0x7f08b01046c0 2024-09-11T08:00:50.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.805+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1077837440 0 0) 0x7f089c003720 con 0x7f08b00fdbc0 2024-09-11T08:00:50.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.805+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f08b01197b0 con 0x7f08b00fdbc0 2024-09-11T08:00:50.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.805+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f08a4001990 con 0x7f08b00fdbc0 2024-09-11T08:00:50.818 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.805+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3754725517 0 0) 0x7f08b011a9b0 con 0x7f08b0100bc0 2024-09-11T08:00:50.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.805+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f089c003720 con 0x7f08b0100bc0 2024-09-11T08:00:50.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.805+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f08ac001a10 con 0x7f08b0100bc0 2024-09-11T08:00:50.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.805+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2935178539 0 0) 0x7f08b011bbb0 con 0x7f08b01046c0 2024-09-11T08:00:50.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.805+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f08b011a9b0 con 0x7f08b01046c0 2024-09-11T08:00:50.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.805+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f08a80017b0 con 0x7f08b01046c0 2024-09-11T08:00:50.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.805+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 <== mon.1 v1:172.21.15.146:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1188515486 0 0) 0x7f08b01197b0 con 0x7f08b00fdbc0 2024-09-11T08:00:50.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.805+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 >> v1:172.21.15.102:6790/0 conn(0x7f08b0100bc0 legacy=0x7f08b01145f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:50.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.805+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 >> v1:172.21.15.102:6789/0 conn(0x7f08b01046c0 legacy=0x7f08b0117eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:50.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.805+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b011cdb0 con 0x7f08b00fdbc0 2024-09-11T08:00:50.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.805+0000 7f08b7d86640 1 -- 172.21.15.146:0/2734067889 --> v1:172.21.15.146:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b011bda0 con 0x7f08b00fdbc0 2024-09-11T08:00:50.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.805+0000 7f08b7d86640 1 -- 172.21.15.146:0/2734067889 --> v1:172.21.15.146:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b011c270 con 0x7f08b00fdbc0 2024-09-11T08:00:50.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.805+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 <== mon.1 v1:172.21.15.146:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f08a4001d20 con 0x7f08b00fdbc0 2024-09-11T08:00:50.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.805+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 <== mon.1 v1:172.21.15.146:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f08a4005bd0 con 0x7f08b00fdbc0 2024-09-11T08:00:50.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.806+0000 7f08b7d86640 1 -- 172.21.15.146:0/2734067889 --> v1:172.21.15.146:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0894005040 con 0x7f08b00fdbc0 2024-09-11T08:00:50.819 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.807+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 <== mon.1 v1:172.21.15.146:6789/0 7 ==== mgrmap(e 16) v1 ==== 90797+0+0 (unknown 1868969210 0 0) 0x7f08a4002270 con 0x7f08b00fdbc0 2024-09-11T08:00:50.820 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.807+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 <== mon.1 v1:172.21.15.146:6789/0 8 ==== osd_map(60..60 src has 1..60) v4 ==== 5938+0+0 (unknown 1150950379 0 0) 0x7f08a4089e10 con 0x7f08b00fdbc0 2024-09-11T08:00:50.820 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.812+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 <== mon.1 v1:172.21.15.146:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f08a4056b60 con 0x7f08b00fdbc0 2024-09-11T08:00:50.994 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.992+0000 7f08b7d86640 1 -- 172.21.15.146:0/2734067889 --> v1:172.21.15.102:6800/2950771867 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi102=a;smithi146=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f0894000bc0 con 0x7f089c06fd80 2024-09-11T08:00:51.001 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:50.999+0000 7f08a27fc640 1 -- 172.21.15.146:0/2734067889 <== mgr.14150 v1:172.21.15.102:6800/2950771867 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (unknown 0 0 240551134) 0x7f0894000bc0 con 0x7f089c06fd80 2024-09-11T08:00:51.001 INFO:teuthology.orchestra.run.smithi146.stdout:Scheduled node-exporter update... 2024-09-11T08:00:51.002 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:51.001+0000 7f08b7d86640 1 -- 172.21.15.146:0/2734067889 >> v1:172.21.15.102:6800/2950771867 conn(0x7f089c06fd80 legacy=0x7f089c072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:51.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:51.001+0000 7f08b7d86640 1 -- 172.21.15.146:0/2734067889 >> v1:172.21.15.146:6789/0 conn(0x7f08b00fdbc0 legacy=0x7f08b00fd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:51.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:51.001+0000 7f08b7d86640 1 -- 172.21.15.146:0/2734067889 shutdown_connections 2024-09-11T08:00:51.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:51.001+0000 7f08b7d86640 1 -- 172.21.15.146:0/2734067889 >> 172.21.15.146:0/2734067889 conn(0x7f08b00f63c0 msgr2=0x7f08b0104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:00:51.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:51.001+0000 7f08b7d86640 1 -- 172.21.15.146:0/2734067889 shutdown_connections 2024-09-11T08:00:51.003 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:51.001+0000 7f08b7d86640 1 -- 172.21.15.146:0/2734067889 wait complete. 2024-09-11T08:00:51.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:51 smithi146 ceph-mon[33158]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T08:00:51.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:51 smithi146 ceph-mon[33158]: from='client.24472 v1:172.21.15.146:0/2734067889' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi102=a;smithi146=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:00:51.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:51 smithi146 ceph-mon[33158]: Saving service node-exporter spec with placement smithi102=a;smithi146=b;count:2 2024-09-11T08:00:51.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:51 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:51.603 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:51 smithi102 ceph-mon[41371]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T08:00:51.603 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:51 smithi102 ceph-mon[41371]: from='client.24472 v1:172.21.15.146:0/2734067889' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi102=a;smithi146=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:00:51.603 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:51 smithi102 ceph-mon[41371]: Saving service node-exporter spec with placement smithi102=a;smithi146=b;count:2 2024-09-11T08:00:51.603 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:51 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:51.603 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:51 smithi102 ceph-mon[29849]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T08:00:51.603 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:51 smithi102 ceph-mon[29849]: from='client.24472 v1:172.21.15.146:0/2734067889' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi102=a;smithi146=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:00:51.603 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:51 smithi102 ceph-mon[29849]: Saving service node-exporter spec with placement smithi102=a;smithi146=b;count:2 2024-09-11T08:00:51.603 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:51 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:51.763 DEBUG:teuthology.orchestra.run.smithi102:node-exporter.a> sudo journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@node-exporter.a.service 2024-09-11T08:00:51.766 DEBUG:teuthology.orchestra.run.smithi146:node-exporter.b> sudo journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@node-exporter.b.service 2024-09-11T08:00:51.768 INFO:tasks.cephadm:Adding alertmanager.a on smithi102 2024-09-11T08:00:51.769 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph orch apply alertmanager '1;smithi102=a' 2024-09-11T08:00:52.190 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:00:52.465 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:52 smithi146 ceph-mon[33158]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.7 KiB/s wr, 141 op/s 2024-09-11T08:00:52.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:52 smithi102 ceph-mon[29849]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.7 KiB/s wr, 141 op/s 2024-09-11T08:00:52.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:52 smithi102 ceph-mon[41371]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.7 KiB/s wr, 141 op/s 2024-09-11T08:00:53.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.862+0000 7f239062e640 1 -- 172.21.15.146:0/4268395620 >> v1:172.21.15.146:6789/0 conn(0x7f23881046d0 legacy=0x7f2388106b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:53.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.862+0000 7f239062e640 1 -- 172.21.15.146:0/4268395620 shutdown_connections 2024-09-11T08:00:53.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.862+0000 7f239062e640 1 -- 172.21.15.146:0/4268395620 >> 172.21.15.146:0/4268395620 conn(0x7f23880f63f0 msgr2=0x7f23880f8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:00:53.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.862+0000 7f239062e640 1 -- 172.21.15.146:0/4268395620 shutdown_connections 2024-09-11T08:00:53.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.863+0000 7f239062e640 1 -- 172.21.15.146:0/4268395620 wait complete. 2024-09-11T08:00:53.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.863+0000 7f239062e640 1 Processor -- start 2024-09-11T08:00:53.895 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.863+0000 7f239062e640 1 -- start start 2024-09-11T08:00:53.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.863+0000 7f239062e640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f23881199b0 con 0x7f23880fdbd0 2024-09-11T08:00:53.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.863+0000 7f239062e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f238811abb0 con 0x7f2388100bd0 2024-09-11T08:00:53.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.863+0000 7f239062e640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f238811bdb0 con 0x7f23881046d0 2024-09-11T08:00:53.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.863+0000 7f238dbc9640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f2388100bd0 0x7f23881147f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:57970/0 (socket says 172.21.15.146:57970) 2024-09-11T08:00:53.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.863+0000 7f238dbc9640 1 -- 172.21.15.146:0/3519091648 learned_addr learned my addr 172.21.15.146:0/3519091648 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-11T08:00:53.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.864+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3192503234 0 0) 0x7f238811abb0 con 0x7f2388100bd0 2024-09-11T08:00:53.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.864+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2374003120 con 0x7f2388100bd0 2024-09-11T08:00:53.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.864+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896313621 0 0) 0x7f23881199b0 con 0x7f23880fdbd0 2024-09-11T08:00:53.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.864+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f238811abb0 con 0x7f23880fdbd0 2024-09-11T08:00:53.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.864+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 980693027 0 0) 0x7f238811bdb0 con 0x7f23881046d0 2024-09-11T08:00:53.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.864+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23881199b0 con 0x7f23881046d0 2024-09-11T08:00:53.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.864+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3865435939 0 0) 0x7f2374003120 con 0x7f2388100bd0 2024-09-11T08:00:53.896 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.864+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f238811bdb0 con 0x7f2388100bd0 2024-09-11T08:00:53.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.864+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f2384001940 con 0x7f2388100bd0 2024-09-11T08:00:53.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.864+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 533682558 0 0) 0x7f238811abb0 con 0x7f23880fdbd0 2024-09-11T08:00:53.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.864+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2374003120 con 0x7f23880fdbd0 2024-09-11T08:00:53.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.864+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f237c0017b0 con 0x7f23880fdbd0 2024-09-11T08:00:53.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.864+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1830015114 0 0) 0x7f23881199b0 con 0x7f23881046d0 2024-09-11T08:00:53.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.864+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f238811abb0 con 0x7f23881046d0 2024-09-11T08:00:53.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.864+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f2380002f80 con 0x7f23881046d0 2024-09-11T08:00:53.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.864+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 <== mon.1 v1:172.21.15.146:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1894948386 0 0) 0x7f238811bdb0 con 0x7f2388100bd0 2024-09-11T08:00:53.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.864+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 >> v1:172.21.15.102:6790/0 conn(0x7f23881046d0 legacy=0x7f23881180b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:53.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.864+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 >> v1:172.21.15.102:6789/0 conn(0x7f23880fdbd0 legacy=0x7f23880fd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:53.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.864+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f238811cfb0 con 0x7f2388100bd0 2024-09-11T08:00:53.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.864+0000 7f239062e640 1 -- 172.21.15.146:0/3519091648 --> v1:172.21.15.146:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2388119ba0 con 0x7f2388100bd0 2024-09-11T08:00:53.897 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.864+0000 7f239062e640 1 -- 172.21.15.146:0/3519091648 --> v1:172.21.15.146:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2388119ff0 con 0x7f2388100bd0 2024-09-11T08:00:53.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.864+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 <== mon.1 v1:172.21.15.146:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f2384001cd0 con 0x7f2388100bd0 2024-09-11T08:00:53.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.864+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 <== mon.1 v1:172.21.15.146:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f2384005bd0 con 0x7f2388100bd0 2024-09-11T08:00:53.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.865+0000 7f239062e640 1 -- 172.21.15.146:0/3519091648 --> v1:172.21.15.146:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236c005040 con 0x7f2388100bd0 2024-09-11T08:00:53.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.865+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 <== mon.1 v1:172.21.15.146:6789/0 7 ==== mgrmap(e 16) v1 ==== 90797+0+0 (unknown 1868969210 0 0) 0x7f2384002270 con 0x7f2388100bd0 2024-09-11T08:00:53.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.866+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 <== mon.1 v1:172.21.15.146:6789/0 8 ==== osd_map(60..60 src has 1..60) v4 ==== 5938+0+0 (unknown 1150950379 0 0) 0x7f2384089d90 con 0x7f2388100bd0 2024-09-11T08:00:53.898 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:53.870+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 <== mon.1 v1:172.21.15.146:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f2384056ac0 con 0x7f2388100bd0 2024-09-11T08:00:54.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:54.055+0000 7f239062e640 1 -- 172.21.15.146:0/3519091648 --> v1:172.21.15.102:6800/2950771867 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi102=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f236c000bc0 con 0x7f237406fea0 2024-09-11T08:00:54.063 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:54.061+0000 7f237affd640 1 -- 172.21.15.146:0/3519091648 <== mgr.14150 v1:172.21.15.102:6800/2950771867 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (unknown 0 0 1850065467) 0x7f236c000bc0 con 0x7f237406fea0 2024-09-11T08:00:54.063 INFO:teuthology.orchestra.run.smithi146.stdout:Scheduled alertmanager update... 2024-09-11T08:00:54.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:54.063+0000 7f239062e640 1 -- 172.21.15.146:0/3519091648 >> v1:172.21.15.102:6800/2950771867 conn(0x7f237406fea0 legacy=0x7f2374072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:54.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:54.063+0000 7f239062e640 1 -- 172.21.15.146:0/3519091648 >> v1:172.21.15.146:6789/0 conn(0x7f2388100bd0 legacy=0x7f23881147f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:54.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:54.063+0000 7f239062e640 1 -- 172.21.15.146:0/3519091648 shutdown_connections 2024-09-11T08:00:54.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:54.063+0000 7f239062e640 1 -- 172.21.15.146:0/3519091648 >> 172.21.15.146:0/3519091648 conn(0x7f23880f63f0 msgr2=0x7f23880f87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:00:54.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:54.063+0000 7f239062e640 1 -- 172.21.15.146:0/3519091648 shutdown_connections 2024-09-11T08:00:54.065 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:54.063+0000 7f239062e640 1 -- 172.21.15.146:0/3519091648 wait complete. 2024-09-11T08:00:54.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:54 smithi102 ceph-mon[41371]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 48 KiB/s rd, 3.7 KiB/s wr, 117 op/s 2024-09-11T08:00:54.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:54 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:54.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:54 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:54.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:54 smithi102 ceph-mon[29849]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 48 KiB/s rd, 3.7 KiB/s wr, 117 op/s 2024-09-11T08:00:54.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:54 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:54.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:54 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:54.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:54 smithi146 ceph-mon[33158]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 48 KiB/s rd, 3.7 KiB/s wr, 117 op/s 2024-09-11T08:00:54.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:54 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:54.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:54 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:55.268 DEBUG:teuthology.orchestra.run.smithi102:alertmanager.a> sudo journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@alertmanager.a.service 2024-09-11T08:00:55.271 INFO:tasks.cephadm:Adding grafana.a on smithi146 2024-09-11T08:00:55.272 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph orch apply grafana '1;smithi146=a' 2024-09-11T08:00:55.527 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:00:55.905 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:55 smithi146 ceph-mon[33158]: from='client.24478 v1:172.21.15.146:0/3519091648' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi102=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:00:55.905 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:55 smithi146 ceph-mon[33158]: Saving service alertmanager spec with placement smithi102=a;count:1 2024-09-11T08:00:55.905 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:55 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:55.905 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:55 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:55.905 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:55 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:55.905 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:55 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:00:55.905 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:55 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:55.906 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:55 smithi146 ceph-mon[33158]: Deploying daemon prometheus.a on smithi146 2024-09-11T08:00:56.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:55 smithi102 ceph-mon[41371]: from='client.24478 v1:172.21.15.146:0/3519091648' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi102=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:00:56.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:55 smithi102 ceph-mon[41371]: Saving service alertmanager spec with placement smithi102=a;count:1 2024-09-11T08:00:56.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:55 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:56.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:55 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:56.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:55 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:56.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:55 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:00:56.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:55 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:56.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:55 smithi102 ceph-mon[41371]: Deploying daemon prometheus.a on smithi146 2024-09-11T08:00:56.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:55 smithi102 ceph-mon[29849]: from='client.24478 v1:172.21.15.146:0/3519091648' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi102=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:00:56.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:55 smithi102 ceph-mon[29849]: Saving service alertmanager spec with placement smithi102=a;count:1 2024-09-11T08:00:56.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:55 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:56.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:55 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:56.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:55 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:00:56.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:55 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:00:56.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:55 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:56.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:55 smithi102 ceph-mon[29849]: Deploying daemon prometheus.a on smithi146 2024-09-11T08:00:57.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:56 smithi146 ceph-mon[33158]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 39 KiB/s rd, 3.1 KiB/s wr, 96 op/s 2024-09-11T08:00:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:56 smithi102 ceph-mon[41371]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 39 KiB/s rd, 3.1 KiB/s wr, 96 op/s 2024-09-11T08:00:57.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:56 smithi102 ceph-mon[29849]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 39 KiB/s rd, 3.1 KiB/s wr, 96 op/s 2024-09-11T08:00:57.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.553+0000 7f220d1e7640 1 -- 172.21.15.146:0/3463138722 >> v1:172.21.15.146:6789/0 conn(0x7f22080ff950 legacy=0x7f2208101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:57.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.553+0000 7f220d1e7640 1 -- 172.21.15.146:0/3463138722 shutdown_connections 2024-09-11T08:00:57.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.553+0000 7f220d1e7640 1 -- 172.21.15.146:0/3463138722 >> 172.21.15.146:0/3463138722 conn(0x7f22080f6c10 msgr2=0x7f22080f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:00:57.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.553+0000 7f220d1e7640 1 -- 172.21.15.146:0/3463138722 shutdown_connections 2024-09-11T08:00:57.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.553+0000 7f220d1e7640 1 -- 172.21.15.146:0/3463138722 wait complete. 2024-09-11T08:00:57.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.554+0000 7f220d1e7640 1 Processor -- start 2024-09-11T08:00:57.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.554+0000 7f220d1e7640 1 -- start start 2024-09-11T08:00:57.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.554+0000 7f220d1e7640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f220810d210 con 0x7f2208103570 2024-09-11T08:00:57.610 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.554+0000 7f220d1e7640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2208118a10 con 0x7f22080fb450 2024-09-11T08:00:57.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.554+0000 7f220d1e7640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2208119bb0 con 0x7f22080ff950 2024-09-11T08:00:57.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.554+0000 7f2206d9d640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f22080fb450 0x7f220810c680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:50140/0 (socket says 172.21.15.146:50140) 2024-09-11T08:00:57.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.554+0000 7f2206d9d640 1 -- 172.21.15.146:0/2715345104 learned_addr learned my addr 172.21.15.146:0/2715345104 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-11T08:00:57.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.554+0000 7f21ef7fe640 1 -- 172.21.15.146:0/2715345104 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1341472109 0 0) 0x7f2208118a10 con 0x7f22080fb450 2024-09-11T08:00:57.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.554+0000 7f21ef7fe640 1 -- 172.21.15.146:0/2715345104 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21f0003120 con 0x7f22080fb450 2024-09-11T08:00:57.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.554+0000 7f21ef7fe640 1 -- 172.21.15.146:0/2715345104 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3921293044 0 0) 0x7f2208119bb0 con 0x7f22080ff950 2024-09-11T08:00:57.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.554+0000 7f21ef7fe640 1 -- 172.21.15.146:0/2715345104 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2208118a10 con 0x7f22080ff950 2024-09-11T08:00:57.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.554+0000 7f21ef7fe640 1 -- 172.21.15.146:0/2715345104 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3125067707 0 0) 0x7f220810d210 con 0x7f2208103570 2024-09-11T08:00:57.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.554+0000 7f21ef7fe640 1 -- 172.21.15.146:0/2715345104 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2208119bb0 con 0x7f2208103570 2024-09-11T08:00:57.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.554+0000 7f21ef7fe640 1 -- 172.21.15.146:0/2715345104 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4213901043 0 0) 0x7f21f0003120 con 0x7f22080fb450 2024-09-11T08:00:57.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.554+0000 7f21ef7fe640 1 -- 172.21.15.146:0/2715345104 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f220810d210 con 0x7f22080fb450 2024-09-11T08:00:57.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.554+0000 7f21ef7fe640 1 -- 172.21.15.146:0/2715345104 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f21f8001940 con 0x7f22080fb450 2024-09-11T08:00:57.611 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.554+0000 7f21ef7fe640 1 -- 172.21.15.146:0/2715345104 <== mon.1 v1:172.21.15.146:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 768359460 0 0) 0x7f220810d210 con 0x7f22080fb450 2024-09-11T08:00:57.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.554+0000 7f21ef7fe640 1 -- 172.21.15.146:0/2715345104 >> v1:172.21.15.102:6790/0 conn(0x7f22080ff950 legacy=0x7f2208112bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:57.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.554+0000 7f21ef7fe640 1 -- 172.21.15.146:0/2715345104 >> v1:172.21.15.102:6789/0 conn(0x7f2208103570 legacy=0x7f22081162e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:57.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.555+0000 7f21ef7fe640 1 -- 172.21.15.146:0/2715345104 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220811ad90 con 0x7f22080fb450 2024-09-11T08:00:57.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.555+0000 7f220d1e7640 1 -- 172.21.15.146:0/2715345104 --> v1:172.21.15.146:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2208119da0 con 0x7f22080fb450 2024-09-11T08:00:57.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.555+0000 7f220d1e7640 1 -- 172.21.15.146:0/2715345104 --> v1:172.21.15.146:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f220811a1f0 con 0x7f22080fb450 2024-09-11T08:00:57.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.555+0000 7f21ef7fe640 1 -- 172.21.15.146:0/2715345104 <== mon.1 v1:172.21.15.146:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f21f8001cd0 con 0x7f22080fb450 2024-09-11T08:00:57.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.555+0000 7f21ef7fe640 1 -- 172.21.15.146:0/2715345104 <== mon.1 v1:172.21.15.146:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f21f8005b60 con 0x7f22080fb450 2024-09-11T08:00:57.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.556+0000 7f220d1e7640 1 -- 172.21.15.146:0/2715345104 --> v1:172.21.15.146:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f4005040 con 0x7f22080fb450 2024-09-11T08:00:57.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.556+0000 7f21ef7fe640 1 -- 172.21.15.146:0/2715345104 <== mon.1 v1:172.21.15.146:6789/0 7 ==== mgrmap(e 16) v1 ==== 90797+0+0 (unknown 1868969210 0 0) 0x7f21f8002270 con 0x7f22080fb450 2024-09-11T08:00:57.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.557+0000 7f21ef7fe640 1 -- 172.21.15.146:0/2715345104 <== mon.1 v1:172.21.15.146:6789/0 8 ==== osd_map(60..60 src has 1..60) v4 ==== 5938+0+0 (unknown 1150950379 0 0) 0x7f21f8089e70 con 0x7f22080fb450 2024-09-11T08:00:57.612 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.561+0000 7f21ef7fe640 1 -- 172.21.15.146:0/2715345104 <== mon.1 v1:172.21.15.146:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f21f8056ae0 con 0x7f22080fb450 2024-09-11T08:00:57.731 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.730+0000 7f220d1e7640 1 -- 172.21.15.146:0/2715345104 --> v1:172.21.15.102:6800/2950771867 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi146=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f21f4000bc0 con 0x7f21f006f990 2024-09-11T08:00:57.738 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.737+0000 7f21ef7fe640 1 -- 172.21.15.146:0/2715345104 <== mgr.14150 v1:172.21.15.102:6800/2950771867 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (unknown 0 0 664801700) 0x7f21f4000bc0 con 0x7f21f006f990 2024-09-11T08:00:57.738 INFO:teuthology.orchestra.run.smithi146.stdout:Scheduled grafana update... 2024-09-11T08:00:57.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.739+0000 7f220d1e7640 1 -- 172.21.15.146:0/2715345104 >> v1:172.21.15.102:6800/2950771867 conn(0x7f21f006f990 legacy=0x7f21f0071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:57.740 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.739+0000 7f220d1e7640 1 -- 172.21.15.146:0/2715345104 >> v1:172.21.15.146:6789/0 conn(0x7f22080fb450 legacy=0x7f220810c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:00:57.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.739+0000 7f220d1e7640 1 -- 172.21.15.146:0/2715345104 shutdown_connections 2024-09-11T08:00:57.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.739+0000 7f220d1e7640 1 -- 172.21.15.146:0/2715345104 >> 172.21.15.146:0/2715345104 conn(0x7f22080f6c10 msgr2=0x7f22080f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:00:57.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.739+0000 7f220d1e7640 1 -- 172.21.15.146:0/2715345104 shutdown_connections 2024-09-11T08:00:57.741 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:00:57.739+0000 7f220d1e7640 1 -- 172.21.15.146:0/2715345104 wait complete. 2024-09-11T08:00:59.078 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:58 smithi146 ceph-mon[33158]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 27 KiB/s rd, 2.0 KiB/s wr, 67 op/s 2024-09-11T08:00:59.078 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:58 smithi146 ceph-mon[33158]: from='client.24484 v1:172.21.15.146:0/2715345104' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi146=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:00:59.078 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:58 smithi146 ceph-mon[33158]: Saving service grafana spec with placement smithi146=a;count:1 2024-09-11T08:00:59.078 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:58 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:59.078 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:00:58 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:59.099 DEBUG:teuthology.orchestra.run.smithi146:grafana.a> sudo journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@grafana.a.service 2024-09-11T08:00:59.101 INFO:tasks.cephadm:Setting up client nodes... 2024-09-11T08:00:59.101 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T08:00:59.127 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:58 smithi102 ceph-mon[41371]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 27 KiB/s rd, 2.0 KiB/s wr, 67 op/s 2024-09-11T08:00:59.127 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:58 smithi102 ceph-mon[41371]: from='client.24484 v1:172.21.15.146:0/2715345104' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi146=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:00:59.127 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:58 smithi102 ceph-mon[41371]: Saving service grafana spec with placement smithi146=a;count:1 2024-09-11T08:00:59.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:58 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:59.128 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:00:58 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:59.128 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:58 smithi102 ceph-mon[29849]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 27 KiB/s rd, 2.0 KiB/s wr, 67 op/s 2024-09-11T08:00:59.128 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:58 smithi102 ceph-mon[29849]: from='client.24484 v1:172.21.15.146:0/2715345104' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi146=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T08:00:59.128 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:58 smithi102 ceph-mon[29849]: Saving service grafana spec with placement smithi146=a;count:1 2024-09-11T08:00:59.128 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:58 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:59.128 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:00:58 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:00:59.349 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:00:59.434 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:00:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:01:00.946 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.944+0000 7f1b86799640 1 -- 172.21.15.102:0/3640971931 >> v1:172.21.15.102:6789/0 conn(0x7f1b801046c0 legacy=0x7f1b80106b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:00.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.944+0000 7f1b86799640 1 -- 172.21.15.102:0/3640971931 shutdown_connections 2024-09-11T08:01:00.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.944+0000 7f1b86799640 1 -- 172.21.15.102:0/3640971931 >> 172.21.15.102:0/3640971931 conn(0x7f1b800f7d40 msgr2=0x7f1b800fa190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:00.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.944+0000 7f1b86799640 1 -- 172.21.15.102:0/3640971931 shutdown_connections 2024-09-11T08:01:00.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.944+0000 7f1b86799640 1 -- 172.21.15.102:0/3640971931 wait complete. 2024-09-11T08:01:00.947 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.945+0000 7f1b86799640 1 Processor -- start 2024-09-11T08:01:00.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.945+0000 7f1b86799640 1 -- start start 2024-09-11T08:01:00.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.945+0000 7f1b86799640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1b80119680 con 0x7f1b801046c0 2024-09-11T08:01:00.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.945+0000 7f1b86799640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1b8011a880 con 0x7f1b80100aa0 2024-09-11T08:01:00.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.945+0000 7f1b86799640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1b8011ba80 con 0x7f1b800fc5a0 2024-09-11T08:01:00.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.945+0000 7f1b7ffff640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7f1b800fc5a0 0x7f1b80071320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:38914/0 (socket says 172.21.15.102:38914) 2024-09-11T08:01:00.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.945+0000 7f1b7ffff640 1 -- 172.21.15.102:0/3634073740 learned_addr learned my addr 172.21.15.102:0/3634073740 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:01:00.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.945+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934436444 0 0) 0x7f1b8011ba80 con 0x7f1b800fc5a0 2024-09-11T08:01:00.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.945+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b680034e0 con 0x7f1b800fc5a0 2024-09-11T08:01:00.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.945+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645017803 0 0) 0x7f1b80119680 con 0x7f1b801046c0 2024-09-11T08:01:00.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.945+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b8011ba80 con 0x7f1b801046c0 2024-09-11T08:01:00.949 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.945+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 415328979 0 0) 0x7f1b8011a880 con 0x7f1b80100aa0 2024-09-11T08:01:00.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.945+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b80119680 con 0x7f1b80100aa0 2024-09-11T08:01:00.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.946+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1736215298 0 0) 0x7f1b680034e0 con 0x7f1b800fc5a0 2024-09-11T08:01:00.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.946+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1b8011a880 con 0x7f1b800fc5a0 2024-09-11T08:01:00.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.946+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f1b70001990 con 0x7f1b800fc5a0 2024-09-11T08:01:00.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.946+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1471201545 0 0) 0x7f1b8011ba80 con 0x7f1b801046c0 2024-09-11T08:01:00.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.946+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1b680034e0 con 0x7f1b801046c0 2024-09-11T08:01:00.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.946+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2566956023 0 0) 0x7f1b80119680 con 0x7f1b80100aa0 2024-09-11T08:01:00.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.946+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1b8011ba80 con 0x7f1b80100aa0 2024-09-11T08:01:00.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.946+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f1b78001360 con 0x7f1b801046c0 2024-09-11T08:01:00.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.946+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f1b740017c0 con 0x7f1b80100aa0 2024-09-11T08:01:00.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.946+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2723915498 0 0) 0x7f1b8011a880 con 0x7f1b800fc5a0 2024-09-11T08:01:00.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.946+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 >> v1:172.21.15.146:6789/0 conn(0x7f1b80100aa0 legacy=0x7f1b801144c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:00.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.946+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 >> v1:172.21.15.102:6789/0 conn(0x7f1b801046c0 legacy=0x7f1b80117d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:00.950 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.946+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8011cc80 con 0x7f1b800fc5a0 2024-09-11T08:01:00.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.946+0000 7f1b86799640 1 -- 172.21.15.102:0/3634073740 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b8011aa70 con 0x7f1b800fc5a0 2024-09-11T08:01:00.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.946+0000 7f1b86799640 1 -- 172.21.15.102:0/3634073740 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8011af40 con 0x7f1b800fc5a0 2024-09-11T08:01:00.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.946+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f1b70002460 con 0x7f1b800fc5a0 2024-09-11T08:01:00.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.946+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f1b70005aa0 con 0x7f1b800fc5a0 2024-09-11T08:01:00.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.947+0000 7f1b86799640 1 -- 172.21.15.102:0/3634073740 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b6c005040 con 0x7f1b800fc5a0 2024-09-11T08:01:00.951 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.947+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 16) v1 ==== 90797+0+0 (unknown 1868969210 0 0) 0x7f1b7001bf80 con 0x7f1b800fc5a0 2024-09-11T08:01:00.952 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.948+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(60..60 src has 1..60) v4 ==== 5938+0+0 (unknown 1150950379 0 0) 0x7f1b70089ea0 con 0x7f1b800fc5a0 2024-09-11T08:01:00.954 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:00.952+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f1b70056b50 con 0x7f1b800fc5a0 2024-09-11T08:01:01.109 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:00 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:01:01.109 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:00 smithi102 ceph-mon[41371]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 1.9 KiB/s wr, 63 op/s 2024-09-11T08:01:01.109 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:00 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:01:01.109 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:00 smithi102 ceph-mon[29849]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 1.9 KiB/s wr, 63 op/s 2024-09-11T08:01:01.171 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:01.169+0000 7f1b86799640 1 -- 172.21.15.102:0/3634073740 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f1b6c005870 con 0x7f1b800fc5a0 2024-09-11T08:01:01.177 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:01.174+0000 7f1b7cff9640 1 -- 172.21.15.102:0/3634073740 <== mon.2 v1:172.21.15.102:6790/0 10 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (unknown 2149379880 0 3488382141) 0x7f1b700564f0 con 0x7f1b800fc5a0 2024-09-11T08:01:01.177 INFO:teuthology.orchestra.run.smithi102.stdout:[client.0] 2024-09-11T08:01:01.177 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQA9TuFmX3JAChAAS8GRzrpJUR2t1/YS21+Oaw== 2024-09-11T08:01:01.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:01.176+0000 7f1b86799640 1 -- 172.21.15.102:0/3634073740 >> v1:172.21.15.102:6800/2950771867 conn(0x7f1b6806fd80 legacy=0x7f1b68072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:01.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:01.177+0000 7f1b86799640 1 -- 172.21.15.102:0/3634073740 >> v1:172.21.15.102:6790/0 conn(0x7f1b800fc5a0 legacy=0x7f1b80071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:01.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:01.177+0000 7f1b86799640 1 -- 172.21.15.102:0/3634073740 shutdown_connections 2024-09-11T08:01:01.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:01.177+0000 7f1b86799640 1 -- 172.21.15.102:0/3634073740 >> 172.21.15.102:0/3634073740 conn(0x7f1b800f7d40 msgr2=0x7f1b80104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:01.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:01.177+0000 7f1b86799640 1 -- 172.21.15.102:0/3634073740 shutdown_connections 2024-09-11T08:01:01.179 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:01.177+0000 7f1b86799640 1 -- 172.21.15.102:0/3634073740 wait complete. 2024-09-11T08:01:01.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:00 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:01:01.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:00 smithi146 ceph-mon[33158]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 1.9 KiB/s wr, 63 op/s 2024-09-11T08:01:01.823 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:01 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3634073740' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:01:01.823 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:01 smithi102 ceph-mon[41371]: from='client.24476 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:01:01.823 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:01 smithi102 ceph-mon[41371]: from='client.24476 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:01:01.823 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:01 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3634073740' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:01:01.823 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:01 smithi102 ceph-mon[29849]: from='client.24476 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:01:01.823 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:01 smithi102 ceph-mon[29849]: from='client.24476 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:01:01.844 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-11T08:01:01.845 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-11T08:01:01.845 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T08:01:01.922 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T08:01:02.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:01 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3634073740' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:01:02.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:01 smithi146 ceph-mon[33158]: from='client.24476 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:01:02.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:01 smithi146 ceph-mon[33158]: from='client.24476 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:01:02.517 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T08:01:02.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:02 smithi146 ceph-mon[33158]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-11T08:01:03.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:02 smithi102 ceph-mon[29849]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-11T08:01:03.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:02 smithi102 ceph-mon[41371]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-11T08:01:04.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:04 smithi146 ceph-mon[33158]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:05.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:04 smithi102 ceph-mon[41371]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:05.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:04 smithi102 ceph-mon[29849]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:05.818 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:05 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.146:0/510397143' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:01:05.818 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:05 smithi146 ceph-mon[33158]: from='client.24496 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:01:05.818 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:05 smithi146 ceph-mon[33158]: from='client.24496 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:01:06.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:05 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.146:0/510397143' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:01:06.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:05 smithi102 ceph-mon[41371]: from='client.24496 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:01:06.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:05 smithi102 ceph-mon[41371]: from='client.24496 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:01:06.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:05 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.146:0/510397143' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:01:06.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:05 smithi102 ceph-mon[29849]: from='client.24496 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T08:01:06.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:05 smithi102 ceph-mon[29849]: from='client.24496 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T08:01:06.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.379+0000 7f1be9c7b640 1 -- 172.21.15.146:0/186798807 >> v1:172.21.15.146:6789/0 conn(0x7f1be4100a90 legacy=0x7f1be4102e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:06.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.379+0000 7f1be9c7b640 1 -- 172.21.15.146:0/186798807 shutdown_connections 2024-09-11T08:01:06.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.379+0000 7f1be9c7b640 1 -- 172.21.15.146:0/186798807 >> 172.21.15.146:0/186798807 conn(0x7f1be40f7d70 msgr2=0x7f1be40fa180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:06.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.379+0000 7f1be9c7b640 1 -- 172.21.15.146:0/186798807 shutdown_connections 2024-09-11T08:01:06.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.379+0000 7f1be9c7b640 1 -- 172.21.15.146:0/186798807 wait complete. 2024-09-11T08:01:06.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.380+0000 7f1be9c7b640 1 Processor -- start 2024-09-11T08:01:06.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.380+0000 7f1be9c7b640 1 -- start start 2024-09-11T08:01:06.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.380+0000 7f1be9c7b640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1be4119940 con 0x7f1be4100a90 2024-09-11T08:01:06.229 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.380+0000 7f1be9c7b640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1be411ab40 con 0x7f1be40fc590 2024-09-11T08:01:06.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.380+0000 7f1be9c7b640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1be411bd40 con 0x7f1be41046b0 2024-09-11T08:01:06.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.380+0000 7f1be37fe640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f1be40fc590 0x7f1be4071320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.146:45644/0 (socket says 172.21.15.146:45644) 2024-09-11T08:01:06.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.380+0000 7f1be37fe640 1 -- 172.21.15.146:0/510397143 learned_addr learned my addr 172.21.15.146:0/510397143 (peer_addr_for_me v1:172.21.15.146:0/0) 2024-09-11T08:01:06.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.380+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 495318122 0 0) 0x7f1be411ab40 con 0x7f1be40fc590 2024-09-11T08:01:06.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.380+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bd00034e0 con 0x7f1be40fc590 2024-09-11T08:01:06.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.380+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2085918724 0 0) 0x7f1be411bd40 con 0x7f1be41046b0 2024-09-11T08:01:06.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.380+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1be411ab40 con 0x7f1be41046b0 2024-09-11T08:01:06.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.380+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 61254180 0 0) 0x7f1be4119940 con 0x7f1be4100a90 2024-09-11T08:01:06.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.380+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1be411bd40 con 0x7f1be4100a90 2024-09-11T08:01:06.230 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.380+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2930370913 0 0) 0x7f1bd00034e0 con 0x7f1be40fc590 2024-09-11T08:01:06.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.381+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1be4119940 con 0x7f1be40fc590 2024-09-11T08:01:06.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.381+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f1bcc001940 con 0x7f1be40fc590 2024-09-11T08:01:06.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.381+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3645643361 0 0) 0x7f1be411ab40 con 0x7f1be41046b0 2024-09-11T08:01:06.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.381+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1bd00034e0 con 0x7f1be41046b0 2024-09-11T08:01:06.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.381+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f1bdc001b80 con 0x7f1be41046b0 2024-09-11T08:01:06.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.381+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 469234842 0 0) 0x7f1be411bd40 con 0x7f1be4100a90 2024-09-11T08:01:06.231 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.381+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1be411ab40 con 0x7f1be4100a90 2024-09-11T08:01:06.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.381+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f1bd4002f90 con 0x7f1be4100a90 2024-09-11T08:01:06.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.381+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 <== mon.1 v1:172.21.15.146:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4063072911 0 0) 0x7f1be4119940 con 0x7f1be40fc590 2024-09-11T08:01:06.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.381+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 >> v1:172.21.15.102:6790/0 conn(0x7f1be41046b0 legacy=0x7f1be4118040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:06.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.381+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 >> v1:172.21.15.102:6789/0 conn(0x7f1be4100a90 legacy=0x7f1be4114780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:06.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.381+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be411cf40 con 0x7f1be40fc590 2024-09-11T08:01:06.232 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.381+0000 7f1be9c7b640 1 -- 172.21.15.146:0/510397143 --> v1:172.21.15.146:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be411bf30 con 0x7f1be40fc590 2024-09-11T08:01:06.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.381+0000 7f1be9c7b640 1 -- 172.21.15.146:0/510397143 --> v1:172.21.15.146:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be411c380 con 0x7f1be40fc590 2024-09-11T08:01:06.233 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.381+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 <== mon.1 v1:172.21.15.146:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f1bcc002410 con 0x7f1be40fc590 2024-09-11T08:01:06.233 INFO:teuthology.orchestra.run.smithi146.stdout:[client.1] 2024-09-11T08:01:06.233 INFO:teuthology.orchestra.run.smithi146.stdout: key = AQBBTuFm8j1xJBAA8QYbwF1ZJuQM6gsj/f9syQ== 2024-09-11T08:01:06.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.381+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 <== mon.1 v1:172.21.15.146:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f1bcc005a80 con 0x7f1be40fc590 2024-09-11T08:01:06.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.382+0000 7f1be9c7b640 1 -- 172.21.15.146:0/510397143 --> v1:172.21.15.146:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc4005040 con 0x7f1be40fc590 2024-09-11T08:01:06.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.382+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 <== mon.1 v1:172.21.15.146:6789/0 7 ==== mgrmap(e 16) v1 ==== 90797+0+0 (unknown 1868969210 0 0) 0x7f1bcc01bf20 con 0x7f1be40fc590 2024-09-11T08:01:06.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.383+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 <== mon.1 v1:172.21.15.146:6789/0 8 ==== osd_map(60..60 src has 1..60) v4 ==== 5938+0+0 (unknown 1150950379 0 0) 0x7f1bcc089ee0 con 0x7f1be40fc590 2024-09-11T08:01:06.234 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.387+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 <== mon.1 v1:172.21.15.146:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f1bcc056710 con 0x7f1be40fc590 2024-09-11T08:01:06.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.609+0000 7f1be9c7b640 1 -- 172.21.15.146:0/510397143 --> v1:172.21.15.146:6789/0 -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f1bc4005870 con 0x7f1be40fc590 2024-09-11T08:01:06.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.614+0000 7f1bdbfff640 1 -- 172.21.15.146:0/510397143 <== mon.1 v1:172.21.15.146:6789/0 10 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (unknown 1363465646 0 1629333780) 0x7f1bcc056530 con 0x7f1be40fc590 2024-09-11T08:01:06.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.617+0000 7f1be9c7b640 1 -- 172.21.15.146:0/510397143 >> v1:172.21.15.102:6800/2950771867 conn(0x7f1bd006feb0 legacy=0x7f1bd0072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:06.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.617+0000 7f1be9c7b640 1 -- 172.21.15.146:0/510397143 >> v1:172.21.15.146:6789/0 conn(0x7f1be40fc590 legacy=0x7f1be4071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:06.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.617+0000 7f1be9c7b640 1 -- 172.21.15.146:0/510397143 shutdown_connections 2024-09-11T08:01:06.235 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.617+0000 7f1be9c7b640 1 -- 172.21.15.146:0/510397143 >> 172.21.15.146:0/510397143 conn(0x7f1be40f7d70 msgr2=0x7f1be4106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:06.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.617+0000 7f1be9c7b640 1 -- 172.21.15.146:0/510397143 shutdown_connections 2024-09-11T08:01:06.236 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-11T08:01:05.617+0000 7f1be9c7b640 1 -- 172.21.15.146:0/510397143 wait complete. 2024-09-11T08:01:06.832 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-11T08:01:06.833 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-11T08:01:06.833 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-11T08:01:06.871 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T08:01:06.871 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-11T08:01:06.871 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-11T08:01:07.059 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:06 smithi146 ceph-mon[33158]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:07.129 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:01:07.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:06 smithi102 ceph-mon[29849]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:07.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:06 smithi102 ceph-mon[41371]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:08.431 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 systemd[1]: Starting Ceph prometheus.a for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:01:08.683 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 podman[64692]: 2024-09-11 08:01:08.430771309 +0000 UTC m=+0.110022639 container create bfcbdfbd6e87e5e3b100f011c0f9c34130198231252a0f18ae145a71d4dead1b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:01:08.683 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 podman[64692]: 2024-09-11 08:01:08.337411546 +0000 UTC m=+0.016662855 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-11T08:01:08.684 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 podman[64692]: 2024-09-11 08:01:08.649355007 +0000 UTC m=+0.328606296 container init bfcbdfbd6e87e5e3b100f011c0f9c34130198231252a0f18ae145a71d4dead1b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:01:08.684 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 podman[64692]: 2024-09-11 08:01:08.653917215 +0000 UTC m=+0.333168513 container start bfcbdfbd6e87e5e3b100f011c0f9c34130198231252a0f18ae145a71d4dead1b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:01:08.684 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:01:08.678Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-11T08:01:08.684 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:01:08.678Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-11T08:01:08.684 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:01:08.678Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi146 (none))" 2024-09-11T08:01:08.684 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:01:08.678Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-11T08:01:08.684 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:01:08.678Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-11T08:01:08.684 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:01:08.679Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-11T08:01:08.684 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:01:08.680Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-11T08:01:08.684 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:01:08.681Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-11T08:01:08.684 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:01:08.681Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-11T08:01:08.684 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:01:08.682Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-11T08:01:08.684 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:01:08.682Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.827µs 2024-09-11T08:01:08.684 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:01:08.682Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-11T08:01:08.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.693+0000 7ff11b551640 1 -- 172.21.15.102:0/3919770642 >> v1:172.21.15.102:6790/0 conn(0x7ff1140fe170 legacy=0x7ff114107620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:08.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.693+0000 7ff11b551640 1 -- 172.21.15.102:0/3919770642 shutdown_connections 2024-09-11T08:01:08.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.693+0000 7ff11b551640 1 -- 172.21.15.102:0/3919770642 >> 172.21.15.102:0/3919770642 conn(0x7ff1140f7000 msgr2=0x7ff1140f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:08.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.693+0000 7ff11b551640 1 -- 172.21.15.102:0/3919770642 shutdown_connections 2024-09-11T08:01:08.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.693+0000 7ff11b551640 1 -- 172.21.15.102:0/3919770642 wait complete. 2024-09-11T08:01:08.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.693+0000 7ff11b551640 1 Processor -- start 2024-09-11T08:01:08.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.693+0000 7ff11b551640 1 -- start start 2024-09-11T08:01:08.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff11b551640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff11411bf90 con 0x7ff1140fd5a0 2024-09-11T08:01:08.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff11b551640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff11411d150 con 0x7ff114108ed0 2024-09-11T08:01:08.699 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff11b551640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff11411e350 con 0x7ff1140fe170 2024-09-11T08:01:08.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff1192ed640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7ff1140fd5a0 0x7ff1141040d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:51734/0 (socket says 172.21.15.102:51734) 2024-09-11T08:01:08.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff1192ed640 1 -- 172.21.15.102:0/3365930872 learned_addr learned my addr 172.21.15.102:0/3365930872 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:01:08.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3133475215 0 0) 0x7ff11411e350 con 0x7ff1140fe170 2024-09-11T08:01:08.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1000034e0 con 0x7ff1140fe170 2024-09-11T08:01:08.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1116257944 0 0) 0x7ff11411bf90 con 0x7ff1140fd5a0 2024-09-11T08:01:08.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff11411e350 con 0x7ff1140fd5a0 2024-09-11T08:01:08.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614046263 0 0) 0x7ff11411d150 con 0x7ff114108ed0 2024-09-11T08:01:08.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff11411bf90 con 0x7ff114108ed0 2024-09-11T08:01:08.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3532804845 0 0) 0x7ff1000034e0 con 0x7ff1140fe170 2024-09-11T08:01:08.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff11411d150 con 0x7ff1140fe170 2024-09-11T08:01:08.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7ff104001430 con 0x7ff1140fe170 2024-09-11T08:01:08.700 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2769439624 0 0) 0x7ff11411bf90 con 0x7ff114108ed0 2024-09-11T08:01:08.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff1000034e0 con 0x7ff114108ed0 2024-09-11T08:01:08.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1872323628 0 0) 0x7ff11411e350 con 0x7ff1140fd5a0 2024-09-11T08:01:08.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff11411bf90 con 0x7ff1140fd5a0 2024-09-11T08:01:08.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7ff10c001bb0 con 0x7ff114108ed0 2024-09-11T08:01:08.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7ff110001900 con 0x7ff1140fd5a0 2024-09-11T08:01:08.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 349134431 0 0) 0x7ff11411d150 con 0x7ff1140fe170 2024-09-11T08:01:08.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 >> v1:172.21.15.146:6789/0 conn(0x7ff114108ed0 legacy=0x7ff11411a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:08.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 >> v1:172.21.15.102:6789/0 conn(0x7ff1140fd5a0 legacy=0x7ff1141040d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:08.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11411f550 con 0x7ff1140fe170 2024-09-11T08:01:08.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.694+0000 7ff11b551640 1 -- 172.21.15.102:0/3365930872 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff11411e540 con 0x7ff1140fe170 2024-09-11T08:01:08.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.695+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7ff104001b30 con 0x7ff1140fe170 2024-09-11T08:01:08.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.695+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7ff1040034e0 con 0x7ff1140fe170 2024-09-11T08:01:08.701 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.695+0000 7ff11b551640 1 -- 172.21.15.102:0/3365930872 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11411e990 con 0x7ff1140fe170 2024-09-11T08:01:08.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.696+0000 7ff11b551640 1 -- 172.21.15.102:0/3365930872 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1140663e0 con 0x7ff1140fe170 2024-09-11T08:01:08.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.696+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 16) v1 ==== 90797+0+0 (unknown 1868969210 0 0) 0x7ff104002050 con 0x7ff1140fe170 2024-09-11T08:01:08.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.697+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(60..60 src has 1..60) v4 ==== 5938+0+0 (unknown 1150950379 0 0) 0x7ff104087630 con 0x7ff1140fe170 2024-09-11T08:01:08.702 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.701+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7ff104054380 con 0x7ff1140fe170 2024-09-11T08:01:08.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.908+0000 7ff11b551640 1 -- 172.21.15.102:0/3365930872 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7ff11404bf60 con 0x7ff1140fe170 2024-09-11T08:01:08.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.910+0000 7ff109ffb640 1 -- 172.21.15.102:0/3365930872 <== mon.2 v1:172.21.15.102:6790/0 10 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v16) v1 ==== 74+0+176664 (unknown 3479712753 0 2029851053) 0x7ff104053d20 con 0x7ff1140fe170 2024-09-11T08:01:08.915 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T08:01:08.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.914+0000 7ff11b551640 1 -- 172.21.15.102:0/3365930872 >> v1:172.21.15.102:6800/2950771867 conn(0x7ff10006fe40 legacy=0x7ff1000722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:08.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.914+0000 7ff11b551640 1 -- 172.21.15.102:0/3365930872 >> v1:172.21.15.102:6790/0 conn(0x7ff1140fe170 legacy=0x7ff1141047e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:08.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.914+0000 7ff11b551640 1 -- 172.21.15.102:0/3365930872 shutdown_connections 2024-09-11T08:01:08.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.914+0000 7ff11b551640 1 -- 172.21.15.102:0/3365930872 >> 172.21.15.102:0/3365930872 conn(0x7ff1140f7000 msgr2=0x7ff1140f93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:08.916 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.914+0000 7ff11b551640 1 -- 172.21.15.102:0/3365930872 shutdown_connections 2024-09-11T08:01:08.917 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:08.914+0000 7ff11b551640 1 -- 172.21.15.102:0/3365930872 wait complete. 2024-09-11T08:01:08.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:08 smithi146 ceph-mon[33158]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:01:08.934 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:01:08.684Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-11T08:01:08.934 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:01:08.684Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=16.697µs wal_replay_duration=1.883382ms wbl_replay_duration=170ns total_replay_duration=1.918221ms 2024-09-11T08:01:08.935 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:01:08.685Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-11T08:01:08.935 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:01:08.685Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-11T08:01:08.935 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:01:08.685Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-11T08:01:08.935 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:01:08.699Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=14.48232ms db_storage=755ns remote_storage=1.151µs web_handler=377ns query_engine=550ns scrape=433.191µs scrape_sd=17.717µs notify=669ns notify_sd=1.378µs rules=13.846391ms tracing=5.845µs 2024-09-11T08:01:08.935 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:01:08.699Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-11T08:01:08.935 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:01:08.700Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-11T08:01:08.935 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 bash[64692]: bfcbdfbd6e87e5e3b100f011c0f9c34130198231252a0f18ae145a71d4dead1b 2024-09-11T08:01:08.935 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:01:08 smithi146 systemd[1]: Started Ceph prometheus.a for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T08:01:09.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:08 smithi102 ceph-mon[41371]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:01:09.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:08 smithi102 ceph-mon[29849]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:01:09.678 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":16,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6800","nonce":2950771867}]},"active_addr":"172.21.15.102:6800/2950771867","active_change":"2024-09-11T07:56:23.399938+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":24109,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.102:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":2698361201}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":557873312}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":2088881890}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":3528827211}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":1389080029}]}]} 2024-09-11T08:01:09.682 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-11T08:01:09.683 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-11T08:01:09.683 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T08:01:09.684 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:01:09 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:01:09.945 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:01:09.959 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:09 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ignoring --setuser ceph since I am not root 2024-09-11T08:01:09.959 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:09 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ignoring --setgroup ceph since I am not root 2024-09-11T08:01:09.960 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:09 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:01:09.960 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:09 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:01:09.960 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:09 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:01:09.960 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:09 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T08:01:09.960 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:09 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3365930872' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T08:01:09.960 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:09 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:01:09.960 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:09 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:01:09.960 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:09 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:01:09.960 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:09 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T08:01:09.960 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:09 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3365930872' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T08:01:10.072 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:09 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:01:10.072 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:09 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:01:10.072 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:09 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' 2024-09-11T08:01:10.072 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:09 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T08:01:10.072 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:09 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3365930872' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T08:01:10.073 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:09 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ignoring --setuser ceph since I am not root 2024-09-11T08:01:10.073 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:09 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ignoring --setgroup ceph since I am not root 2024-09-11T08:01:10.073 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:09 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:09.950+0000 7fe14057bec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:01:10.073 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:10 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:10.005+0000 7fe14057bec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:01:10.073 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:10 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:10.070+0000 7fe14057bec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:01:10.215 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:09 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:09.958+0000 7f9342aacec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T08:01:10.215 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:10 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:10.017+0000 7f9342aacec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T08:01:10.215 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:10 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:10.084+0000 7f9342aacec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T08:01:10.215 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:10 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:10.213+0000 7f9342aacec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:01:10.333 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:10 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:10.195+0000 7fe14057bec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:01:10.588 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:10 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:10.331+0000 7fe14057bec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:01:10.622 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:10 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:10.361+0000 7f9342aacec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T08:01:10.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:10 smithi102 ceph-mon[29849]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T08:01:10.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:10 smithi102 ceph-mon[29849]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T08:01:10.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:10 smithi102 ceph-mon[41371]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T08:01:10.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:10 smithi102 ceph-mon[41371]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T08:01:10.896 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:10 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:10.621+0000 7f9342aacec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:01:10.896 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:10 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:10.681+0000 7f9342aacec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:01:10.896 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:10 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:10.849+0000 7f9342aacec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:01:10.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:10 smithi146 ceph-mon[33158]: from='mgr.14150 v1:172.21.15.102:0/1382255631' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T08:01:10.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:10 smithi146 ceph-mon[33158]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T08:01:10.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:10 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:10.585+0000 7fe14057bec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T08:01:10.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:10 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:10.645+0000 7fe14057bec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T08:01:10.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:10 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:10.806+0000 7fe14057bec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T08:01:11.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.633+0000 7fdc772fa640 1 -- 172.21.15.102:0/723559830 >> v1:172.21.15.102:6790/0 conn(0x7fdc700fe180 legacy=0x7fdc70107630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:11.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.633+0000 7fdc772fa640 1 -- 172.21.15.102:0/723559830 shutdown_connections 2024-09-11T08:01:11.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.633+0000 7fdc772fa640 1 -- 172.21.15.102:0/723559830 >> 172.21.15.102:0/723559830 conn(0x7fdc700752c0 msgr2=0x7fdc700f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:11.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.634+0000 7fdc772fa640 1 -- 172.21.15.102:0/723559830 shutdown_connections 2024-09-11T08:01:11.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.634+0000 7fdc772fa640 1 -- 172.21.15.102:0/723559830 wait complete. 2024-09-11T08:01:11.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.634+0000 7fdc772fa640 1 Processor -- start 2024-09-11T08:01:11.656 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.634+0000 7fdc772fa640 1 -- start start 2024-09-11T08:01:11.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc772fa640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdc7011bf20 con 0x7fdc700fe180 2024-09-11T08:01:11.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc772fa640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdc7011d120 con 0x7fdc700fd5b0 2024-09-11T08:01:11.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc772fa640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdc7011e320 con 0x7fdc70108f10 2024-09-11T08:01:11.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc74895640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fdc700fe180 0x7fdc701047e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:51768/0 (socket says 172.21.15.102:51768) 2024-09-11T08:01:11.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc74895640 1 -- 172.21.15.102:0/446069945 learned_addr learned my addr 172.21.15.102:0/446069945 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:01:11.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 136674364 0 0) 0x7fdc7011e320 con 0x7fdc70108f10 2024-09-11T08:01:11.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc5c0034e0 con 0x7fdc70108f10 2024-09-11T08:01:11.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1983035454 0 0) 0x7fdc7011bf20 con 0x7fdc700fe180 2024-09-11T08:01:11.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc7011e320 con 0x7fdc700fe180 2024-09-11T08:01:11.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2864132349 0 0) 0x7fdc7011d120 con 0x7fdc700fd5b0 2024-09-11T08:01:11.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc7011bf20 con 0x7fdc700fd5b0 2024-09-11T08:01:11.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2424838416 0 0) 0x7fdc5c0034e0 con 0x7fdc70108f10 2024-09-11T08:01:11.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdc7011d120 con 0x7fdc70108f10 2024-09-11T08:01:11.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fdc68001c40 con 0x7fdc70108f10 2024-09-11T08:01:11.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2103233160 0 0) 0x7fdc7011e320 con 0x7fdc700fe180 2024-09-11T08:01:11.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdc5c0034e0 con 0x7fdc700fe180 2024-09-11T08:01:11.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fdc60002aa0 con 0x7fdc700fe180 2024-09-11T08:01:11.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3403742132 0 0) 0x7fdc7011bf20 con 0x7fdc700fd5b0 2024-09-11T08:01:11.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdc7011e320 con 0x7fdc700fd5b0 2024-09-11T08:01:11.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fdc6c001830 con 0x7fdc700fd5b0 2024-09-11T08:01:11.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1128954024 0 0) 0x7fdc7011d120 con 0x7fdc70108f10 2024-09-11T08:01:11.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 >> v1:172.21.15.146:6789/0 conn(0x7fdc700fd5b0 legacy=0x7fdc701040d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:11.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 >> v1:172.21.15.102:6789/0 conn(0x7fdc700fe180 legacy=0x7fdc701047e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:11.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7011f520 con 0x7fdc70108f10 2024-09-11T08:01:11.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.635+0000 7fdc772fa640 1 -- 172.21.15.102:0/446069945 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc7011c110 con 0x7fdc70108f10 2024-09-11T08:01:11.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.636+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7fdc68002380 con 0x7fdc70108f10 2024-09-11T08:01:11.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.636+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fdc68003540 con 0x7fdc70108f10 2024-09-11T08:01:11.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.636+0000 7fdc772fa640 1 -- 172.21.15.102:0/446069945 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc7011c560 con 0x7fdc70108f10 2024-09-11T08:01:11.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.636+0000 7fdc772fa640 1 -- 172.21.15.102:0/446069945 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc700656f0 con 0x7fdc70108f10 2024-09-11T08:01:11.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.637+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 17) v1 ==== 90811+0+0 (unknown 2283475751 0 0) 0x7fdc68001f30 con 0x7fdc70108f10 2024-09-11T08:01:11.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.637+0000 7fdc75096640 1 -- 172.21.15.102:0/446069945 >> v1:172.21.15.102:6800/2950771867 conn(0x7fdc5c06fdc0 legacy=0x7fdc5c072270 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.102:6800/2950771867 2024-09-11T08:01:11.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.638+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(60..60 src has 1..60) v4 ==== 5938+0+0 (unknown 1150950379 0 0) 0x7fdc68087920 con 0x7fdc70108f10 2024-09-11T08:01:11.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.641+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fdc68054650 con 0x7fdc70108f10 2024-09-11T08:01:11.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.806+0000 7fdc772fa640 1 -- 172.21.15.102:0/446069945 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc701053c0 con 0x7fdc70108f10 2024-09-11T08:01:11.808 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.807+0000 7fdc65ffb640 1 -- 172.21.15.102:0/446069945 <== mon.2 v1:172.21.15.102:6790/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+18923 (unknown 2432737747 0 963460098) 0x7fdc68053ff0 con 0x7fdc70108f10 2024-09-11T08:01:11.809 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T08:01:11.809 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":60,"fsid":"2051892c-7013-11ef-bcea-c7b262605968","created":"2024-09-11T07:55:31.241924+0000","modified":"2024-09-11T08:00:46.138100+0000","last_up_change":"2024-09-11T08:00:23.529399+0000","last_in_change":"2024-09-11T08:00:07.275908+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T07:58:29.695819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-11T08:00:35.371973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-11T08:00:35.606310+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":56,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T08:00:37.093623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T08:00:39.055311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T08:00:41.136996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"08248393-1eca-4c59-b577-e0ee4ed4d6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":444496098}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":444496098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":444496098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":444496098}]},"public_addr":"172.21.15.102:6801/444496098","cluster_addr":"172.21.15.102:6802/444496098","heartbeat_back_addr":"172.21.15.102:6804/444496098","heartbeat_front_addr":"172.21.15.102:6803/444496098","state":["exists","up"]},{"osd":1,"uuid":"209cc4e5-6ff0-49f4-8ce4-0540ea1b5c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":2501991548}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":2501991548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":2501991548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":2501991548}]},"public_addr":"172.21.15.102:6805/2501991548","cluster_addr":"172.21.15.102:6806/2501991548","heartbeat_back_addr":"172.21.15.102:6808/2501991548","heartbeat_front_addr":"172.21.15.102:6807/2501991548","state":["exists","up"]},{"osd":2,"uuid":"e900567f-3dca-4cde-abbb-d8fa53cec635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":3842496257}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":3842496257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":3842496257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":3842496257}]},"public_addr":"172.21.15.102:6809/3842496257","cluster_addr":"172.21.15.102:6810/3842496257","heartbeat_back_addr":"172.21.15.102:6812/3842496257","heartbeat_front_addr":"172.21.15.102:6811/3842496257","state":["exists","up"]},{"osd":3,"uuid":"c0d77f9c-0e1b-45b5-8b9b-e84097814fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":426874071}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":426874071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6816","nonce":426874071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":426874071}]},"public_addr":"172.21.15.102:6813/426874071","cluster_addr":"172.21.15.102:6814/426874071","heartbeat_back_addr":"172.21.15.102:6816/426874071","heartbeat_front_addr":"172.21.15.102:6815/426874071","state":["exists","up"]},{"osd":4,"uuid":"31d92bc5-10be-47d0-b91c-5e5496c25ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6800","nonce":2908355826}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6801","nonce":2908355826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6803","nonce":2908355826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6802","nonce":2908355826}]},"public_addr":"172.21.15.146:6800/2908355826","cluster_addr":"172.21.15.146:6801/2908355826","heartbeat_back_addr":"172.21.15.146:6803/2908355826","heartbeat_front_addr":"172.21.15.146:6802/2908355826","state":["exists","up"]},{"osd":5,"uuid":"6272822a-2ca3-45c8-a3c6-c00e26753989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6804","nonce":1497668584}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6805","nonce":1497668584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6807","nonce":1497668584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6806","nonce":1497668584}]},"public_addr":"172.21.15.146:6804/1497668584","cluster_addr":"172.21.15.146:6805/1497668584","heartbeat_back_addr":"172.21.15.146:6807/1497668584","heartbeat_front_addr":"172.21.15.146:6806/1497668584","state":["exists","up"]},{"osd":6,"uuid":"1c3a857f-a8d5-46d0-8fd5-bcc759074f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6808","nonce":85686736}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6809","nonce":85686736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6811","nonce":85686736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6810","nonce":85686736}]},"public_addr":"172.21.15.146:6808/85686736","cluster_addr":"172.21.15.146:6809/85686736","heartbeat_back_addr":"172.21.15.146:6811/85686736","heartbeat_front_addr":"172.21.15.146:6810/85686736","state":["exists","up"]},{"osd":7,"uuid":"ca7e36e9-ca16-4d15-8264-deed759fdbf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6812","nonce":2181809960}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6813","nonce":2181809960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6815","nonce":2181809960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6814","nonce":2181809960}]},"public_addr":"172.21.15.146:6812/2181809960","cluster_addr":"172.21.15.146:6813/2181809960","heartbeat_back_addr":"172.21.15.146:6815/2181809960","heartbeat_front_addr":"172.21.15.146:6814/2181809960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:57:40.079569+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:58:03.521245+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:58:27.496621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:58:50.615763+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:59:11.931460+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:59:35.063555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:59:58.367568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:00:22.451943+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:0/2854313951":"2024-09-12T07:56:23.399794+0000","172.21.15.102:0/4190790614":"2024-09-12T07:56:23.399794+0000","172.21.15.102:0/3435847265":"2024-09-12T07:56:23.399794+0000","172.21.15.102:0/234741464":"2024-09-12T07:56:01.048720+0000","172.21.15.102:6800/3158655170":"2024-09-12T07:56:01.048720+0000","172.21.15.102:6800/3294079105":"2024-09-12T07:56:23.399794+0000","172.21.15.102:0/3790980491":"2024-09-12T07:56:01.048720+0000","172.21.15.102:0/831066729":"2024-09-12T07:56:01.048720+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:01:11.810 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.809+0000 7fdc772fa640 1 -- 172.21.15.102:0/446069945 >> v1:172.21.15.102:6800/2950771867 conn(0x7fdc5c06fdc0 legacy=0x7fdc5c072270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T08:01:11.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.809+0000 7fdc772fa640 1 -- 172.21.15.102:0/446069945 >> v1:172.21.15.102:6790/0 conn(0x7fdc70108f10 legacy=0x7fdc7011a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:11.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.809+0000 7fdc772fa640 1 -- 172.21.15.102:0/446069945 shutdown_connections 2024-09-11T08:01:11.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.809+0000 7fdc772fa640 1 -- 172.21.15.102:0/446069945 >> 172.21.15.102:0/446069945 conn(0x7fdc700752c0 msgr2=0x7fdc700f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:11.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.810+0000 7fdc772fa640 1 -- 172.21.15.102:0/446069945 shutdown_connections 2024-09-11T08:01:11.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:11.810+0000 7fdc772fa640 1 -- 172.21.15.102:0/446069945 wait complete. 2024-09-11T08:01:11.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:11 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/446069945' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:01:11.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:11.655+0000 7fe14057bec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:01:11.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:11.720+0000 7fe14057bec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:01:12.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:11 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/446069945' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:01:12.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:11 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/446069945' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:01:12.147 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:11 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:11.725+0000 7f9342aacec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T08:01:12.147 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:11 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:11.791+0000 7f9342aacec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T08:01:12.535 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-11T08:01:12.536 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T08:01:12.684 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:12 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:12.422+0000 7fe14057bec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:01:12.684 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:12 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:12.532+0000 7fe14057bec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:01:12.684 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:12 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:12.652+0000 7fe14057bec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:01:12.791 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:01:12.825 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:12 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:12.505+0000 7f9342aacec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T08:01:12.825 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:12 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:12.625+0000 7f9342aacec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T08:01:12.825 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:12 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:12.755+0000 7f9342aacec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T08:01:12.825 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:12 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:12.824+0000 7f9342aacec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:01:12.966 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:12 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:12.711+0000 7fe14057bec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T08:01:12.967 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:12 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:12.838+0000 7fe14057bec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:01:12.967 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:12 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:12.907+0000 7fe14057bec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:01:13.085 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:12 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:12.954+0000 7f9342aacec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T08:01:13.086 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:13 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:13.021+0000 7f9342aacec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T08:01:13.086 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:13 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:13.084+0000 7f9342aacec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:01:13.230 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:12 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:12.965+0000 7fe14057bec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T08:01:13.231 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:13 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:13.100+0000 7fe14057bec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:01:13.231 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:13 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:13.164+0000 7fe14057bec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:01:13.353 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:13 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:13.221+0000 7f9342aacec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T08:01:13.353 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:13 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:13.285+0000 7f9342aacec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T08:01:13.353 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:13 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:13.352+0000 7f9342aacec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:01:13.642 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:13 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:13.433+0000 7f9342aacec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:01:13.642 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:13 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:13.490+0000 7f9342aacec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:01:13.642 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:13 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:13.547+0000 7f9342aacec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:01:13.684 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:13 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:13.229+0000 7fe14057bec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T08:01:13.684 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:13 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:13.310+0000 7fe14057bec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T08:01:13.684 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:13 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:13.367+0000 7fe14057bec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T08:01:13.684 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:13 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:13.423+0000 7fe14057bec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T08:01:14.372 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:14 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:14.313+0000 7f9342aacec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:01:14.372 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:14 smithi102 ceph-mon[29849]: Standby manager daemon x restarted 2024-09-11T08:01:14.372 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:14 smithi102 ceph-mon[29849]: Standby manager daemon x started 2024-09-11T08:01:14.373 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:14 smithi102 ceph-mon[29849]: from='mgr.? v1:172.21.15.146:0/1444757805' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:01:14.373 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:14 smithi102 ceph-mon[29849]: from='mgr.? v1:172.21.15.146:0/1444757805' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:01:14.373 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:14 smithi102 ceph-mon[29849]: from='mgr.? v1:172.21.15.146:0/1444757805' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:01:14.373 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:14 smithi102 ceph-mon[29849]: from='mgr.? v1:172.21.15.146:0/1444757805' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:01:14.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:14 smithi102 ceph-mon[41371]: Standby manager daemon x restarted 2024-09-11T08:01:14.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:14 smithi102 ceph-mon[41371]: Standby manager daemon x started 2024-09-11T08:01:14.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:14 smithi102 ceph-mon[41371]: from='mgr.? v1:172.21.15.146:0/1444757805' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:01:14.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:14 smithi102 ceph-mon[41371]: from='mgr.? v1:172.21.15.146:0/1444757805' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:01:14.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:14 smithi102 ceph-mon[41371]: from='mgr.? v1:172.21.15.146:0/1444757805' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:01:14.373 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:14 smithi102 ceph-mon[41371]: from='mgr.? v1:172.21.15.146:0/1444757805' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:01:14.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:14 smithi146 ceph-mon[33158]: Standby manager daemon x restarted 2024-09-11T08:01:14.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:14 smithi146 ceph-mon[33158]: Standby manager daemon x started 2024-09-11T08:01:14.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:14 smithi146 ceph-mon[33158]: from='mgr.? v1:172.21.15.146:0/1444757805' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T08:01:14.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:14 smithi146 ceph-mon[33158]: from='mgr.? v1:172.21.15.146:0/1444757805' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T08:01:14.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:14 smithi146 ceph-mon[33158]: from='mgr.? v1:172.21.15.146:0/1444757805' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T08:01:14.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:14 smithi146 ceph-mon[33158]: from='mgr.? v1:172.21.15.146:0/1444757805' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T08:01:14.434 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:14.171+0000 7fe14057bec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T08:01:14.435 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: [11/Sep/2024:08:01:14] ENGINE Bus STARTING 2024-09-11T08:01:14.435 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: CherryPy Checker: 2024-09-11T08:01:14.435 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: The Application mounted at '' has an empty config. 2024-09-11T08:01:14.435 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: 2024-09-11T08:01:14.435 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: [11/Sep/2024:08:01:14] ENGINE Serving on http://:::9283 2024-09-11T08:01:14.435 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: [11/Sep/2024:08:01:14] ENGINE Bus STARTED 2024-09-11T08:01:14.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.590+0000 7fdef228e640 1 -- 172.21.15.102:0/305981112 >> v1:172.21.15.102:6789/0 conn(0x7fdeec107e50 legacy=0x7fdeec10a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:14.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.590+0000 7fdef228e640 1 -- 172.21.15.102:0/305981112 shutdown_connections 2024-09-11T08:01:14.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.590+0000 7fdef228e640 1 -- 172.21.15.102:0/305981112 >> 172.21.15.102:0/305981112 conn(0x7fdeec074a50 msgr2=0x7fdeec0f8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:14.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.590+0000 7fdef228e640 1 -- 172.21.15.102:0/305981112 shutdown_connections 2024-09-11T08:01:14.607 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.590+0000 7fdef228e640 1 -- 172.21.15.102:0/305981112 wait complete. 2024-09-11T08:01:14.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.590+0000 7fdef228e640 1 Processor -- start 2024-09-11T08:01:14.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.590+0000 7fdef228e640 1 -- start start 2024-09-11T08:01:14.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.591+0000 7fdef228e640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdeec1227c0 con 0x7fdeec107e50 2024-09-11T08:01:14.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.591+0000 7fdef228e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdeec123960 con 0x7fdeec1042f0 2024-09-11T08:01:14.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.591+0000 7fdeebfff640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7fdeec101580 0x7fdeec0faa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:57548/0 (socket says 172.21.15.102:57548) 2024-09-11T08:01:14.608 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.591+0000 7fdeebfff640 1 -- 172.21.15.102:0/2496023093 learned_addr learned my addr 172.21.15.102:0/2496023093 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:01:14.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.591+0000 7fdef228e640 1 -- 172.21.15.102:0/2496023093 --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdeec124b00 con 0x7fdeec101580 2024-09-11T08:01:14.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.591+0000 7fdee8ff9640 1 -- 172.21.15.102:0/2496023093 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007454120 0 0) 0x7fdeec124b00 con 0x7fdeec101580 2024-09-11T08:01:14.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.591+0000 7fdee8ff9640 1 -- 172.21.15.102:0/2496023093 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fded40034e0 con 0x7fdeec101580 2024-09-11T08:01:14.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.591+0000 7fdee8ff9640 1 -- 172.21.15.102:0/2496023093 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2798835634 0 0) 0x7fdeec123960 con 0x7fdeec1042f0 2024-09-11T08:01:14.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.591+0000 7fdee8ff9640 1 -- 172.21.15.102:0/2496023093 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdeec124b00 con 0x7fdeec1042f0 2024-09-11T08:01:14.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.591+0000 7fdee8ff9640 1 -- 172.21.15.102:0/2496023093 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3712309111 0 0) 0x7fded40034e0 con 0x7fdeec101580 2024-09-11T08:01:14.609 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.591+0000 7fdee8ff9640 1 -- 172.21.15.102:0/2496023093 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdeec123960 con 0x7fdeec101580 2024-09-11T08:01:14.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.591+0000 7fdee8ff9640 1 -- 172.21.15.102:0/2496023093 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fdedc001950 con 0x7fdeec101580 2024-09-11T08:01:14.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.591+0000 7fdee8ff9640 1 -- 172.21.15.102:0/2496023093 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3120715687 0 0) 0x7fdeec124b00 con 0x7fdeec1042f0 2024-09-11T08:01:14.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.591+0000 7fdee8ff9640 1 -- 172.21.15.102:0/2496023093 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fded40034e0 con 0x7fdeec1042f0 2024-09-11T08:01:14.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.591+0000 7fdee8ff9640 1 -- 172.21.15.102:0/2496023093 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fdee0002870 con 0x7fdeec1042f0 2024-09-11T08:01:14.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.592+0000 7fdee8ff9640 1 -- 172.21.15.102:0/2496023093 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3015568596 0 0) 0x7fdeec123960 con 0x7fdeec101580 2024-09-11T08:01:14.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.592+0000 7fdee8ff9640 1 -- 172.21.15.102:0/2496023093 >> v1:172.21.15.146:6789/0 conn(0x7fdeec1042f0 legacy=0x7fdeec0fb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:14.610 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.592+0000 7fdee8ff9640 1 -- 172.21.15.102:0/2496023093 >> v1:172.21.15.102:6789/0 conn(0x7fdeec107e50 legacy=0x7fdeec1210a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:14.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.592+0000 7fdee8ff9640 1 -- 172.21.15.102:0/2496023093 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeec125ca0 con 0x7fdeec101580 2024-09-11T08:01:14.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.592+0000 7fdef228e640 1 -- 172.21.15.102:0/2496023093 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeec123b50 con 0x7fdeec101580 2024-09-11T08:01:14.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.592+0000 7fdef228e640 1 -- 172.21.15.102:0/2496023093 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeec123fa0 con 0x7fdeec101580 2024-09-11T08:01:14.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.592+0000 7fdee8ff9640 1 -- 172.21.15.102:0/2496023093 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7fdedc002470 con 0x7fdeec101580 2024-09-11T08:01:14.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.592+0000 7fdee8ff9640 1 -- 172.21.15.102:0/2496023093 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fdedc005a40 con 0x7fdeec101580 2024-09-11T08:01:14.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.593+0000 7fdef228e640 1 -- 172.21.15.102:0/2496023093 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeec0fcdc0 con 0x7fdeec101580 2024-09-11T08:01:14.611 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.593+0000 7fdee8ff9640 1 -- 172.21.15.102:0/2496023093 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 19) v1 ==== 90462+0+0 (unknown 3778353700 0 0) 0x7fdedc01bd10 con 0x7fdeec101580 2024-09-11T08:01:14.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.594+0000 7fdee8ff9640 1 -- 172.21.15.102:0/2496023093 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 376332686 0 0) 0x7fdedc089890 con 0x7fdeec101580 2024-09-11T08:01:14.612 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.598+0000 7fdee8ff9640 1 -- 172.21.15.102:0/2496023093 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fdedc053270 con 0x7fdeec101580 2024-09-11T08:01:14.646 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:14 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:01:14] ENGINE Bus STARTING 2024-09-11T08:01:14.646 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:14 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: CherryPy Checker: 2024-09-11T08:01:14.646 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:14 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: The Application mounted at '' has an empty config. 2024-09-11T08:01:14.646 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:14 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:14.646 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:14 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:01:14] ENGINE Serving on http://:::9283 2024-09-11T08:01:14.646 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:14 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:01:14] ENGINE Bus STARTED 2024-09-11T08:01:14.788 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.786+0000 7fdef228e640 1 -- 172.21.15.102:0/2496023093 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdeec04bf60 con 0x7fdeec101580 2024-09-11T08:01:14.789 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.788+0000 7fdee8ff9640 1 -- 172.21.15.102:0/2496023093 <== mon.2 v1:172.21.15.102:6790/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+19260 (unknown 1877459060 0 414223474) 0x7fdedc056380 con 0x7fdeec101580 2024-09-11T08:01:14.789 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T08:01:14.790 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":61,"fsid":"2051892c-7013-11ef-bcea-c7b262605968","created":"2024-09-11T07:55:31.241924+0000","modified":"2024-09-11T08:01:14.315778+0000","last_up_change":"2024-09-11T08:00:23.529399+0000","last_in_change":"2024-09-11T08:00:07.275908+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T07:58:29.695819+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-11T08:00:35.371973+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-11T08:00:35.606310+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":56,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T08:00:37.093623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T08:00:39.055311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T08:00:41.136996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"08248393-1eca-4c59-b577-e0ee4ed4d6e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6801","nonce":444496098}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6802","nonce":444496098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6804","nonce":444496098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6803","nonce":444496098}]},"public_addr":"172.21.15.102:6801/444496098","cluster_addr":"172.21.15.102:6802/444496098","heartbeat_back_addr":"172.21.15.102:6804/444496098","heartbeat_front_addr":"172.21.15.102:6803/444496098","state":["exists","up"]},{"osd":1,"uuid":"209cc4e5-6ff0-49f4-8ce4-0540ea1b5c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6805","nonce":2501991548}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6806","nonce":2501991548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6808","nonce":2501991548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6807","nonce":2501991548}]},"public_addr":"172.21.15.102:6805/2501991548","cluster_addr":"172.21.15.102:6806/2501991548","heartbeat_back_addr":"172.21.15.102:6808/2501991548","heartbeat_front_addr":"172.21.15.102:6807/2501991548","state":["exists","up"]},{"osd":2,"uuid":"e900567f-3dca-4cde-abbb-d8fa53cec635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6809","nonce":3842496257}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6810","nonce":3842496257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6812","nonce":3842496257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6811","nonce":3842496257}]},"public_addr":"172.21.15.102:6809/3842496257","cluster_addr":"172.21.15.102:6810/3842496257","heartbeat_back_addr":"172.21.15.102:6812/3842496257","heartbeat_front_addr":"172.21.15.102:6811/3842496257","state":["exists","up"]},{"osd":3,"uuid":"c0d77f9c-0e1b-45b5-8b9b-e84097814fb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6813","nonce":426874071}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6814","nonce":426874071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6816","nonce":426874071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.102:6815","nonce":426874071}]},"public_addr":"172.21.15.102:6813/426874071","cluster_addr":"172.21.15.102:6814/426874071","heartbeat_back_addr":"172.21.15.102:6816/426874071","heartbeat_front_addr":"172.21.15.102:6815/426874071","state":["exists","up"]},{"osd":4,"uuid":"31d92bc5-10be-47d0-b91c-5e5496c25ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6800","nonce":2908355826}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6801","nonce":2908355826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6803","nonce":2908355826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6802","nonce":2908355826}]},"public_addr":"172.21.15.146:6800/2908355826","cluster_addr":"172.21.15.146:6801/2908355826","heartbeat_back_addr":"172.21.15.146:6803/2908355826","heartbeat_front_addr":"172.21.15.146:6802/2908355826","state":["exists","up"]},{"osd":5,"uuid":"6272822a-2ca3-45c8-a3c6-c00e26753989","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6804","nonce":1497668584}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6805","nonce":1497668584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6807","nonce":1497668584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6806","nonce":1497668584}]},"public_addr":"172.21.15.146:6804/1497668584","cluster_addr":"172.21.15.146:6805/1497668584","heartbeat_back_addr":"172.21.15.146:6807/1497668584","heartbeat_front_addr":"172.21.15.146:6806/1497668584","state":["exists","up"]},{"osd":6,"uuid":"1c3a857f-a8d5-46d0-8fd5-bcc759074f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6808","nonce":85686736}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6809","nonce":85686736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6811","nonce":85686736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6810","nonce":85686736}]},"public_addr":"172.21.15.146:6808/85686736","cluster_addr":"172.21.15.146:6809/85686736","heartbeat_back_addr":"172.21.15.146:6811/85686736","heartbeat_front_addr":"172.21.15.146:6810/85686736","state":["exists","up"]},{"osd":7,"uuid":"ca7e36e9-ca16-4d15-8264-deed759fdbf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6812","nonce":2181809960}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6813","nonce":2181809960}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6815","nonce":2181809960}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.146:6814","nonce":2181809960}]},"public_addr":"172.21.15.146:6812/2181809960","cluster_addr":"172.21.15.146:6813/2181809960","heartbeat_back_addr":"172.21.15.146:6815/2181809960","heartbeat_front_addr":"172.21.15.146:6814/2181809960","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:57:40.079569+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:58:03.521245+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:58:27.496621+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:58:50.615763+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:59:11.931460+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:59:35.063555+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:59:58.367568+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T08:00:22.451943+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.102:6800/3294079105":"2024-09-12T07:56:23.399794+0000","172.21.15.102:6800/2950771867":"2024-09-12T08:01:14.315749+0000","172.21.15.102:0/557873312":"2024-09-12T08:01:14.315749+0000","172.21.15.102:0/3790980491":"2024-09-12T07:56:01.048720+0000","172.21.15.102:0/234741464":"2024-09-12T07:56:01.048720+0000","172.21.15.102:0/831066729":"2024-09-12T07:56:01.048720+0000","172.21.15.102:0/3435847265":"2024-09-12T07:56:23.399794+0000","172.21.15.102:0/2854313951":"2024-09-12T07:56:23.399794+0000","172.21.15.102:0/2088881890":"2024-09-12T08:01:14.315749+0000","172.21.15.102:6800/3158655170":"2024-09-12T07:56:01.048720+0000","172.21.15.102:0/4190790614":"2024-09-12T07:56:23.399794+0000","172.21.15.102:0/3528827211":"2024-09-12T08:01:14.315749+0000","172.21.15.102:0/2698361201":"2024-09-12T08:01:14.315749+0000","172.21.15.102:0/1389080029":"2024-09-12T08:01:14.315749+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T08:01:14.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.790+0000 7fdef228e640 1 -- 172.21.15.102:0/2496023093 >> v1:172.21.15.102:6790/0 conn(0x7fdeec101580 legacy=0x7fdeec0faa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:14.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.790+0000 7fdef228e640 1 -- 172.21.15.102:0/2496023093 shutdown_connections 2024-09-11T08:01:14.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.790+0000 7fdef228e640 1 -- 172.21.15.102:0/2496023093 >> 172.21.15.102:0/2496023093 conn(0x7fdeec074a50 msgr2=0x7fdeec10ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:14.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.790+0000 7fdef228e640 1 -- 172.21.15.102:0/2496023093 shutdown_connections 2024-09-11T08:01:14.792 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:14.790+0000 7fdef228e640 1 -- 172.21.15.102:0/2496023093 wait complete. 2024-09-11T08:01:14.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:01:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:01:15.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-11T08:01:15.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: Active manager daemon y restarted 2024-09-11T08:01:15.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: Activating manager daemon y 2024-09-11T08:01:15.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T08:01:15.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: mgrmap e19: y(active, starting, since 0.00718218s), standbys: x 2024-09-11T08:01:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:01:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:01:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:01:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:01:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:01:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:01:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:01:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:01:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:01:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:01:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:01:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:01:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:01:15.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:01:15.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:01:15.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:01:15.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: Manager daemon y is now available 2024-09-11T08:01:15.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:01:15.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:15.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:01:15.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:01:15.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:01:15.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:01:15.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:01:15.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2496023093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:01:15.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-11T08:01:15.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: Active manager daemon y restarted 2024-09-11T08:01:15.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: Activating manager daemon y 2024-09-11T08:01:15.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T08:01:15.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: mgrmap e19: y(active, starting, since 0.00718218s), standbys: x 2024-09-11T08:01:15.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:01:15.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:01:15.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:01:15.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:01:15.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:01:15.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:01:15.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:01:15.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:01:15.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:01:15.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:01:15.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:01:15.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:01:15.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:01:15.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:01:15.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:01:15.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:01:15.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: Manager daemon y is now available 2024-09-11T08:01:15.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:01:15.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:15.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:01:15.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:01:15.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:01:15.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:01:15.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:01:15.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2496023093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:01:15.504 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-11T08:01:15.504 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: Active manager daemon y restarted 2024-09-11T08:01:15.504 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: Activating manager daemon y 2024-09-11T08:01:15.504 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T08:01:15.505 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: mgrmap e19: y(active, starting, since 0.00718218s), standbys: x 2024-09-11T08:01:15.505 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T08:01:15.505 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T08:01:15.505 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T08:01:15.505 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T08:01:15.505 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T08:01:15.505 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T08:01:15.505 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T08:01:15.505 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T08:01:15.505 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T08:01:15.505 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T08:01:15.505 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T08:01:15.505 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T08:01:15.506 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T08:01:15.506 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T08:01:15.506 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T08:01:15.506 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T08:01:15.506 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: Manager daemon y is now available 2024-09-11T08:01:15.506 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:01:15.506 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:15.506 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:01:15.506 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:01:15.506 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T08:01:15.506 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:01:15.506 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T08:01:15.506 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:15 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2496023093' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T08:01:15.509 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-11T08:01:15.509 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-11T08:01:15.510 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-11T08:01:15.510 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-11T08:01:15.511 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-11T08:01:15.511 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-11T08:01:15.511 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-11T08:01:15.512 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-11T08:01:15.896 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:15.437+0000 7f92e586e640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:08:01:15] ENGINE Error in HTTPServer.serve 2024-09-11T08:01:15.897 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: Traceback (most recent call last): 2024-09-11T08:01:15.897 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:01:15.897 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: self._connections.run(self.expiration_interval) 2024-09-11T08:01:15.897 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:01:15.897 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: self._run(expiration_interval) 2024-09-11T08:01:15.897 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:01:15.897 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:01:15.897 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:01:15.898 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:01:15.898 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:01:15.898 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: s = self.context.wrap_socket( 2024-09-11T08:01:15.898 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:01:15.898 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: return self.sslsocket_class._create( 2024-09-11T08:01:15.898 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:01:15.898 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: self.do_handshake() 2024-09-11T08:01:15.898 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:01:15.899 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: self._sslobj.do_handshake() 2024-09-11T08:01:15.899 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:01:15.899 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:01:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:16 smithi102 ceph-mon[29849]: [11/Sep/2024:08:01:15] ENGINE Bus STARTING 2024-09-11T08:01:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:16 smithi102 ceph-mon[29849]: mgrmap e20: y(active, since 1.01045s), standbys: x 2024-09-11T08:01:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:16 smithi102 ceph-mon[29849]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:01:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:16 smithi102 ceph-mon[29849]: [11/Sep/2024:08:01:15] ENGINE Serving on https://172.21.15.102:7150 2024-09-11T08:01:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:16 smithi102 ceph-mon[29849]: [11/Sep/2024:08:01:15] ENGINE Bus STARTED 2024-09-11T08:01:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:16 smithi102 ceph-mon[29849]: [11/Sep/2024:08:01:15] ENGINE Error in HTTPServer.serve 2024-09-11T08:01:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout: Traceback (most recent call last): 2024-09-11T08:01:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:01:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:01:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:01:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout: self._run(expiration_interval) 2024-09-11T08:01:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:01:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:01:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:01:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:01:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:01:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout: s = self.context.wrap_socket( 2024-09-11T08:01:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:01:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout: return self.sslsocket_class._create( 2024-09-11T08:01:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:01:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout: self.do_handshake() 2024-09-11T08:01:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:01:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout: self._sslobj.do_handshake() 2024-09-11T08:01:16.648 INFO:journalctl@ceph.mon.a.smithi102.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:01:16.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:16 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:16.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:16 smithi102 ceph-mon[41371]: [11/Sep/2024:08:01:15] ENGINE Bus STARTING 2024-09-11T08:01:16.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:16 smithi102 ceph-mon[41371]: mgrmap e20: y(active, since 1.01045s), standbys: x 2024-09-11T08:01:16.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:16 smithi102 ceph-mon[41371]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:01:16.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:16 smithi102 ceph-mon[41371]: [11/Sep/2024:08:01:15] ENGINE Serving on https://172.21.15.102:7150 2024-09-11T08:01:16.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:16 smithi102 ceph-mon[41371]: [11/Sep/2024:08:01:15] ENGINE Bus STARTED 2024-09-11T08:01:16.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:16 smithi102 ceph-mon[41371]: [11/Sep/2024:08:01:15] ENGINE Error in HTTPServer.serve 2024-09-11T08:01:16.648 INFO:journalctl@ceph.mon.c.smithi102.stdout: Traceback (most recent call last): 2024-09-11T08:01:16.648 INFO:journalctl@ceph.mon.c.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:01:16.649 INFO:journalctl@ceph.mon.c.smithi102.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:01:16.649 INFO:journalctl@ceph.mon.c.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:01:16.649 INFO:journalctl@ceph.mon.c.smithi102.stdout: self._run(expiration_interval) 2024-09-11T08:01:16.649 INFO:journalctl@ceph.mon.c.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:01:16.649 INFO:journalctl@ceph.mon.c.smithi102.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:01:16.649 INFO:journalctl@ceph.mon.c.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:01:16.649 INFO:journalctl@ceph.mon.c.smithi102.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:01:16.649 INFO:journalctl@ceph.mon.c.smithi102.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:01:16.649 INFO:journalctl@ceph.mon.c.smithi102.stdout: s = self.context.wrap_socket( 2024-09-11T08:01:16.649 INFO:journalctl@ceph.mon.c.smithi102.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:01:16.649 INFO:journalctl@ceph.mon.c.smithi102.stdout: return self.sslsocket_class._create( 2024-09-11T08:01:16.649 INFO:journalctl@ceph.mon.c.smithi102.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:01:16.649 INFO:journalctl@ceph.mon.c.smithi102.stdout: self.do_handshake() 2024-09-11T08:01:16.650 INFO:journalctl@ceph.mon.c.smithi102.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:01:16.650 INFO:journalctl@ceph.mon.c.smithi102.stdout: self._sslobj.do_handshake() 2024-09-11T08:01:16.650 INFO:journalctl@ceph.mon.c.smithi102.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:01:16.650 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:16 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:16.667 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:01:16.672 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:01:16.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:16 smithi146 ceph-mon[33158]: [11/Sep/2024:08:01:15] ENGINE Bus STARTING 2024-09-11T08:01:16.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:16 smithi146 ceph-mon[33158]: mgrmap e20: y(active, since 1.01045s), standbys: x 2024-09-11T08:01:16.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:16 smithi146 ceph-mon[33158]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:01:16.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:16 smithi146 ceph-mon[33158]: [11/Sep/2024:08:01:15] ENGINE Serving on https://172.21.15.102:7150 2024-09-11T08:01:16.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:16 smithi146 ceph-mon[33158]: [11/Sep/2024:08:01:15] ENGINE Bus STARTED 2024-09-11T08:01:16.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:16 smithi146 ceph-mon[33158]: [11/Sep/2024:08:01:15] ENGINE Error in HTTPServer.serve 2024-09-11T08:01:16.685 INFO:journalctl@ceph.mon.b.smithi146.stdout: Traceback (most recent call last): 2024-09-11T08:01:16.685 INFO:journalctl@ceph.mon.b.smithi146.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T08:01:16.685 INFO:journalctl@ceph.mon.b.smithi146.stdout: self._connections.run(self.expiration_interval) 2024-09-11T08:01:16.685 INFO:journalctl@ceph.mon.b.smithi146.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T08:01:16.685 INFO:journalctl@ceph.mon.b.smithi146.stdout: self._run(expiration_interval) 2024-09-11T08:01:16.685 INFO:journalctl@ceph.mon.b.smithi146.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T08:01:16.685 INFO:journalctl@ceph.mon.b.smithi146.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T08:01:16.685 INFO:journalctl@ceph.mon.b.smithi146.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T08:01:16.686 INFO:journalctl@ceph.mon.b.smithi146.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T08:01:16.686 INFO:journalctl@ceph.mon.b.smithi146.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T08:01:16.686 INFO:journalctl@ceph.mon.b.smithi146.stdout: s = self.context.wrap_socket( 2024-09-11T08:01:16.686 INFO:journalctl@ceph.mon.b.smithi146.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T08:01:16.686 INFO:journalctl@ceph.mon.b.smithi146.stdout: return self.sslsocket_class._create( 2024-09-11T08:01:16.686 INFO:journalctl@ceph.mon.b.smithi146.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T08:01:16.686 INFO:journalctl@ceph.mon.b.smithi146.stdout: self.do_handshake() 2024-09-11T08:01:16.686 INFO:journalctl@ceph.mon.b.smithi146.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T08:01:16.686 INFO:journalctl@ceph.mon.b.smithi146.stdout: self._sslobj.do_handshake() 2024-09-11T08:01:16.686 INFO:journalctl@ceph.mon.b.smithi146.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T08:01:16.686 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:16 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:16.709 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:01:16.717 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:01:17.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:17 smithi102 ceph-mon[29849]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:01:17.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:17 smithi102 ceph-mon[29849]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-11T08:01:17.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:17 smithi102 ceph-mon[41371]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:01:17.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:17 smithi102 ceph-mon[41371]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-11T08:01:17.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:17 smithi146 ceph-mon[33158]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:01:17.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:17 smithi146 ceph-mon[33158]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-11T08:01:18.603 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:18.603 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:18.603 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:18.603 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:18.603 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.603 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.603 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.603 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.603 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.603 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.603 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.604 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.604 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:18.604 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:18.604 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:18.604 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:18.604 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:18.604 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.604 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.604 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.604 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.604 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.604 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.604 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.605 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.605 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:18 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:18.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:18 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:18.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:18 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:18.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:18 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:18.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:18 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:18.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:18 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:18 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:18 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:18 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:18 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:18 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:18 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:18 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:18.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:18 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:19.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:19 smithi146 ceph-mon[33158]: Adjusting osd_memory_target on smithi146 to 3760M 2024-09-11T08:01:19.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:19 smithi146 ceph-mon[33158]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:01:19.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:19 smithi146 ceph-mon[33158]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-11T08:01:19.684 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:01:19 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:01:19.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:19 smithi102 ceph-mon[41371]: Adjusting osd_memory_target on smithi146 to 3760M 2024-09-11T08:01:19.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:19 smithi102 ceph-mon[41371]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:01:19.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:19 smithi102 ceph-mon[41371]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-11T08:01:19.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:19 smithi102 ceph-mon[29849]: Adjusting osd_memory_target on smithi146 to 3760M 2024-09-11T08:01:19.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:19 smithi102 ceph-mon[29849]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:01:19.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:19 smithi102 ceph-mon[29849]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-11T08:01:20.158 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:01:20.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:20 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:01:20.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:20 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:01:20.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.650+0000 7f2cfd7b9640 1 -- 172.21.15.102:0/2921015465 >> v1:172.21.15.102:6790/0 conn(0x7f2cf8108ea0 legacy=0x7f2cf810b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:20.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.651+0000 7f2cfd7b9640 1 -- 172.21.15.102:0/2921015465 shutdown_connections 2024-09-11T08:01:20.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.651+0000 7f2cfd7b9640 1 -- 172.21.15.102:0/2921015465 >> 172.21.15.102:0/2921015465 conn(0x7f2cf80749e0 msgr2=0x7f2cf80f8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:20.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.651+0000 7f2cfd7b9640 1 -- 172.21.15.102:0/2921015465 shutdown_connections 2024-09-11T08:01:20.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.651+0000 7f2cfd7b9640 1 -- 172.21.15.102:0/2921015465 wait complete. 2024-09-11T08:01:20.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.651+0000 7f2cfd7b9640 1 Processor -- start 2024-09-11T08:01:20.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.651+0000 7f2cfd7b9640 1 -- start start 2024-09-11T08:01:20.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.651+0000 7f2cfd7b9640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2cf811bf80 con 0x7f2cf8102590 2024-09-11T08:01:20.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.651+0000 7f2cfd7b9640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2cf811d140 con 0x7f2cf8105370 2024-09-11T08:01:20.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.651+0000 7f2cfd7b9640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2cf811e340 con 0x7f2cf8108ea0 2024-09-11T08:01:20.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.651+0000 7f2cf6ffd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f2cf8102590 0x7f2cf80fe5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46752/0 (socket says 172.21.15.102:46752) 2024-09-11T08:01:20.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.651+0000 7f2cf6ffd640 1 -- 172.21.15.102:0/1027172886 learned_addr learned my addr 172.21.15.102:0/1027172886 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:01:20.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.652+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 319314390 0 0) 0x7f2cf811e340 con 0x7f2cf8108ea0 2024-09-11T08:01:20.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.652+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ce4003120 con 0x7f2cf8108ea0 2024-09-11T08:01:20.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.652+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2173786182 0 0) 0x7f2cf811bf80 con 0x7f2cf8102590 2024-09-11T08:01:20.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.652+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cf811e340 con 0x7f2cf8102590 2024-09-11T08:01:20.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.652+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 652451766 0 0) 0x7f2cf811d140 con 0x7f2cf8105370 2024-09-11T08:01:20.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.652+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cf811bf80 con 0x7f2cf8105370 2024-09-11T08:01:20.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.652+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 558893890 0 0) 0x7f2ce4003120 con 0x7f2cf8108ea0 2024-09-11T08:01:20.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.652+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2cf811d140 con 0x7f2cf8108ea0 2024-09-11T08:01:20.675 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.652+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f2cf0001ce0 con 0x7f2cf8108ea0 2024-09-11T08:01:20.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.652+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2194801000 0 0) 0x7f2cf811e340 con 0x7f2cf8102590 2024-09-11T08:01:20.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.652+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2ce4003120 con 0x7f2cf8102590 2024-09-11T08:01:20.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.652+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f2ce8001940 con 0x7f2cf8102590 2024-09-11T08:01:20.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.652+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4201871587 0 0) 0x7f2cf811bf80 con 0x7f2cf8105370 2024-09-11T08:01:20.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.652+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2cf811e340 con 0x7f2cf8105370 2024-09-11T08:01:20.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.652+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f2ce00019f0 con 0x7f2cf8105370 2024-09-11T08:01:20.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.652+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3243754814 0 0) 0x7f2cf811d140 con 0x7f2cf8108ea0 2024-09-11T08:01:20.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.652+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 >> v1:172.21.15.146:6789/0 conn(0x7f2cf8105370 legacy=0x7f2cf80fecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:20.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.652+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 >> v1:172.21.15.102:6789/0 conn(0x7f2cf8102590 legacy=0x7f2cf80fe5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:20.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.652+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf811f540 con 0x7f2cf8108ea0 2024-09-11T08:01:20.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.652+0000 7f2cfd7b9640 1 -- 172.21.15.102:0/1027172886 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf811c170 con 0x7f2cf8108ea0 2024-09-11T08:01:20.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.652+0000 7f2cfd7b9640 1 -- 172.21.15.102:0/1027172886 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf811c5c0 con 0x7f2cf8108ea0 2024-09-11T08:01:20.676 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.652+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f2cf0002670 con 0x7f2cf8108ea0 2024-09-11T08:01:20.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.652+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f2cf0003880 con 0x7f2cf8108ea0 2024-09-11T08:01:20.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.653+0000 7f2cfd7b9640 1 -- 172.21.15.102:0/1027172886 --> v1:172.21.15.102:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2cd8000f20 con 0x7f2cf8108ea0 2024-09-11T08:01:20.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.653+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7f2cf00021f0 con 0x7f2cf8108ea0 2024-09-11T08:01:20.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.654+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 376332686 0 0) 0x7f2cf0087ba0 con 0x7f2cf8108ea0 2024-09-11T08:01:20.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.654+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 --> v1:172.21.15.102:6801/444496098 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ce407e250 con 0x7f2ce407a5b0 2024-09-11T08:01:20.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.654+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (unknown 1962166362 0 0) 0x7f2cf0087fc0 con 0x7f2cf8108ea0 2024-09-11T08:01:20.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.655+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 <== osd.0 v1:172.21.15.102:6801/444496098 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f2ce407e250 con 0x7f2ce407a5b0 2024-09-11T08:01:20.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.667+0000 7f2cfd7b9640 1 -- 172.21.15.102:0/1027172886 --> v1:172.21.15.102:6801/444496098 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2cd8002b10 con 0x7f2ce407a5b0 2024-09-11T08:01:20.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.667+0000 7f2cef7fe640 1 -- 172.21.15.102:0/1027172886 <== osd.0 v1:172.21.15.102:6801/444496098 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1996119316) 0x7f2cd8002b10 con 0x7f2ce407a5b0 2024-09-11T08:01:20.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.668+0000 7f2cfd7b9640 1 -- 172.21.15.102:0/1027172886 >> v1:172.21.15.102:6801/444496098 conn(0x7f2ce407a5b0 legacy=0x7f2ce407c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:20.677 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.668+0000 7f2cfd7b9640 1 -- 172.21.15.102:0/1027172886 >> v1:172.21.15.102:6800/4281557748 conn(0x7f2ce406fda0 legacy=0x7f2ce4072250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:20.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.668+0000 7f2cfd7b9640 1 -- 172.21.15.102:0/1027172886 >> v1:172.21.15.102:6790/0 conn(0x7f2cf8108ea0 legacy=0x7f2cf811a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:20.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.668+0000 7f2cf77fe640 1 -- 172.21.15.102:0/1027172886 reap_dead start 2024-09-11T08:01:20.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.668+0000 7f2cfd7b9640 1 -- 172.21.15.102:0/1027172886 shutdown_connections 2024-09-11T08:01:20.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.668+0000 7f2cfd7b9640 1 -- 172.21.15.102:0/1027172886 >> 172.21.15.102:0/1027172886 conn(0x7f2cf80749e0 msgr2=0x7f2cf80f8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:20.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.668+0000 7f2cfd7b9640 1 -- 172.21.15.102:0/1027172886 shutdown_connections 2024-09-11T08:01:20.678 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:20.668+0000 7f2cfd7b9640 1 -- 172.21.15.102:0/1027172886 wait complete. 2024-09-11T08:01:20.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:20 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:01:21.362 INFO:teuthology.orchestra.run.smithi102.stdout:34359738414 2024-09-11T08:01:21.362 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-11T08:01:21.593 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:21 smithi102 ceph-mon[41371]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1 KiB/s rd, 1 op/s 2024-09-11T08:01:21.594 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:21 smithi102 ceph-mon[29849]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1 KiB/s rd, 1 op/s 2024-09-11T08:01:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:21 smithi146 ceph-mon[33158]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1 KiB/s rd, 1 op/s 2024-09-11T08:01:22.733 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:22.734 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:22.734 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:22.734 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:22.734 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.734 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.734 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.734 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.734 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.734 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.734 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.734 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.735 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:22.735 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:01:22.735 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:01:22.735 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:22.735 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:22.735 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:22.735 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:22.735 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.735 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.735 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.735 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.735 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.736 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.736 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.736 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.736 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:22.736 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:01:22.736 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:22 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:01:22.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:22 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:22.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:22 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:22.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:22 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:22.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:22 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:22.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:22 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:22 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:22 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:22 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:22 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:22 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:22 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:22 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:01:22.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:22 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:22.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:22 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:01:22.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:22 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:01:23.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.652+0000 7f823c2af640 1 -- 172.21.15.102:0/3496182278 >> v1:172.21.15.102:6790/0 conn(0x7f8234102610 legacy=0x7f82341029e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:23.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.652+0000 7f823c2af640 1 -- 172.21.15.102:0/3496182278 shutdown_connections 2024-09-11T08:01:23.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.652+0000 7f823c2af640 1 -- 172.21.15.102:0/3496182278 >> 172.21.15.102:0/3496182278 conn(0x7f82340749e0 msgr2=0x7f82340f8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:23.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.652+0000 7f823c2af640 1 -- 172.21.15.102:0/3496182278 shutdown_connections 2024-09-11T08:01:23.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.652+0000 7f823c2af640 1 -- 172.21.15.102:0/3496182278 wait complete. 2024-09-11T08:01:23.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.653+0000 7f823c2af640 1 Processor -- start 2024-09-11T08:01:23.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.653+0000 7f823c2af640 1 -- start start 2024-09-11T08:01:23.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.653+0000 7f823c2af640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f823411c0a0 con 0x7f8234102610 2024-09-11T08:01:23.657 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.653+0000 7f823c2af640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f823411d260 con 0x7f8234108ee0 2024-09-11T08:01:23.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.653+0000 7f823c2af640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f823411e460 con 0x7f8234105380 2024-09-11T08:01:23.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.653+0000 7f823a04b640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f8234102610 0x7f82340fe5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:46770/0 (socket says 172.21.15.102:46770) 2024-09-11T08:01:23.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.653+0000 7f823a04b640 1 -- 172.21.15.102:0/3384246661 learned_addr learned my addr 172.21.15.102:0/3384246661 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:01:23.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.654+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2755980177 0 0) 0x7f823411d260 con 0x7f8234108ee0 2024-09-11T08:01:23.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.654+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82200034e0 con 0x7f8234108ee0 2024-09-11T08:01:23.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.654+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2923417793 0 0) 0x7f823411e460 con 0x7f8234105380 2024-09-11T08:01:23.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.654+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f823411d260 con 0x7f8234105380 2024-09-11T08:01:23.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.654+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1915717520 0 0) 0x7f823411c0a0 con 0x7f8234102610 2024-09-11T08:01:23.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.654+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f823411e460 con 0x7f8234102610 2024-09-11T08:01:23.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.654+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1847978093 0 0) 0x7f82200034e0 con 0x7f8234108ee0 2024-09-11T08:01:23.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.654+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f823411c0a0 con 0x7f8234108ee0 2024-09-11T08:01:23.658 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.654+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f822c0019a0 con 0x7f8234108ee0 2024-09-11T08:01:23.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.654+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 711861867 0 0) 0x7f823411d260 con 0x7f8234105380 2024-09-11T08:01:23.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.654+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f82200034e0 con 0x7f8234105380 2024-09-11T08:01:23.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.654+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f8224001840 con 0x7f8234105380 2024-09-11T08:01:23.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.654+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 <== mon.1 v1:172.21.15.146:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1627908704 0 0) 0x7f823411c0a0 con 0x7f8234108ee0 2024-09-11T08:01:23.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.654+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 >> v1:172.21.15.102:6790/0 conn(0x7f8234105380 legacy=0x7f82340fed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:23.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.654+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 >> v1:172.21.15.102:6789/0 conn(0x7f8234102610 legacy=0x7f82340fe5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:23.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.654+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823411f660 con 0x7f8234108ee0 2024-09-11T08:01:23.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.654+0000 7f823c2af640 1 -- 172.21.15.102:0/3384246661 --> v1:172.21.15.146:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f823411d450 con 0x7f8234108ee0 2024-09-11T08:01:23.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.654+0000 7f823c2af640 1 -- 172.21.15.102:0/3384246661 --> v1:172.21.15.146:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f823411d8a0 con 0x7f8234108ee0 2024-09-11T08:01:23.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.654+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 <== mon.1 v1:172.21.15.146:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f822c002530 con 0x7f8234108ee0 2024-09-11T08:01:23.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.654+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 <== mon.1 v1:172.21.15.146:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f822c003370 con 0x7f8234108ee0 2024-09-11T08:01:23.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.655+0000 7f823c2af640 1 -- 172.21.15.102:0/3384246661 --> v1:172.21.15.146:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8218000f20 con 0x7f8234108ee0 2024-09-11T08:01:23.659 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.656+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 <== mon.1 v1:172.21.15.146:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7f822c003570 con 0x7f8234108ee0 2024-09-11T08:01:23.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.656+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 <== mon.1 v1:172.21.15.146:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 376332686 0 0) 0x7f822c087760 con 0x7f8234108ee0 2024-09-11T08:01:23.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.656+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 --> v1:172.21.15.102:6805/2501991548 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f82200824f0 con 0x7f822007e850 2024-09-11T08:01:23.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.656+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 <== mon.1 v1:172.21.15.146:6789/0 9 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (unknown 1962166362 0 0) 0x7f822c087b90 con 0x7f8234108ee0 2024-09-11T08:01:23.660 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.657+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 <== osd.1 v1:172.21.15.102:6805/2501991548 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f82200824f0 con 0x7f822007e850 2024-09-11T08:01:23.673 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.671+0000 7f823c2af640 1 -- 172.21.15.102:0/3384246661 --> v1:172.21.15.102:6805/2501991548 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8218002b10 con 0x7f822007e850 2024-09-11T08:01:23.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.671+0000 7f822affd640 1 -- 172.21.15.102:0/3384246661 <== osd.1 v1:172.21.15.102:6805/2501991548 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 459644397) 0x7f8218002b10 con 0x7f822007e850 2024-09-11T08:01:23.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.672+0000 7f823c2af640 1 -- 172.21.15.102:0/3384246661 >> v1:172.21.15.102:6805/2501991548 conn(0x7f822007e850 legacy=0x7f8220080c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:23.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.672+0000 7f823c2af640 1 -- 172.21.15.102:0/3384246661 >> v1:172.21.15.102:6800/4281557748 conn(0x7f8220073f50 legacy=0x7f8220076400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:23.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.672+0000 7f823c2af640 1 -- 172.21.15.102:0/3384246661 >> v1:172.21.15.146:6789/0 conn(0x7f8234108ee0 legacy=0x7f823411a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:23.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.672+0000 7f823a84c640 1 -- 172.21.15.102:0/3384246661 reap_dead start 2024-09-11T08:01:23.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.672+0000 7f823c2af640 1 -- 172.21.15.102:0/3384246661 shutdown_connections 2024-09-11T08:01:23.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.672+0000 7f823c2af640 1 -- 172.21.15.102:0/3384246661 >> 172.21.15.102:0/3384246661 conn(0x7f82340749e0 msgr2=0x7f82340f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:23.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.672+0000 7f823c2af640 1 -- 172.21.15.102:0/3384246661 shutdown_connections 2024-09-11T08:01:23.674 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.672+0000 7f823c2af640 1 -- 172.21.15.102:0/3384246661 wait complete. 2024-09-11T08:01:23.863 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:23 smithi102 ceph-mon[29849]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T08:01:23.863 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:23 smithi102 ceph-mon[29849]: Adjusting osd_memory_target on smithi102 to 3760M 2024-09-11T08:01:23.863 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:23 smithi102 ceph-mon[29849]: Updating smithi102:/etc/ceph/ceph.conf 2024-09-11T08:01:23.863 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:23 smithi102 ceph-mon[29849]: Updating smithi146:/etc/ceph/ceph.conf 2024-09-11T08:01:23.863 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:23 smithi102 ceph-mon[29849]: Updating smithi102:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:01:23.863 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:23 smithi102 ceph-mon[29849]: Updating smithi146:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:01:23.863 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:23 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:23.864 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:23 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:23.864 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:23 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:23.864 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:23 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:23.864 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:23 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:23.864 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:23 smithi102 ceph-mon[41371]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T08:01:23.864 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:23 smithi102 ceph-mon[41371]: Adjusting osd_memory_target on smithi102 to 3760M 2024-09-11T08:01:23.864 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:23 smithi102 ceph-mon[41371]: Updating smithi102:/etc/ceph/ceph.conf 2024-09-11T08:01:23.864 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:23 smithi102 ceph-mon[41371]: Updating smithi146:/etc/ceph/ceph.conf 2024-09-11T08:01:23.864 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:23 smithi102 ceph-mon[41371]: Updating smithi102:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:01:23.864 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:23 smithi102 ceph-mon[41371]: Updating smithi146:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:01:23.865 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:23 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:23.865 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:23 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:23.865 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:23 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:23.865 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:23 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:23.865 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:23 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:23.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:23 smithi146 ceph-mon[33158]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T08:01:23.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:23 smithi146 ceph-mon[33158]: Adjusting osd_memory_target on smithi102 to 3760M 2024-09-11T08:01:23.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:23 smithi146 ceph-mon[33158]: Updating smithi102:/etc/ceph/ceph.conf 2024-09-11T08:01:23.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:23 smithi146 ceph-mon[33158]: Updating smithi146:/etc/ceph/ceph.conf 2024-09-11T08:01:23.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:23 smithi146 ceph-mon[33158]: Updating smithi102:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:01:23.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:23 smithi146 ceph-mon[33158]: Updating smithi146:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T08:01:23.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:23 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:23.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:23 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:23.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:23 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:23.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:23 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:23.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:23 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:24.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.861+0000 7f07349ee640 1 -- 172.21.15.102:0/741074280 >> v1:172.21.15.102:6789/0 conn(0x7f0730102610 legacy=0x7f07301029e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:24.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.861+0000 7f07349ee640 1 -- 172.21.15.102:0/741074280 shutdown_connections 2024-09-11T08:01:24.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.861+0000 7f07349ee640 1 -- 172.21.15.102:0/741074280 >> 172.21.15.102:0/741074280 conn(0x7f07300749e0 msgr2=0x7f07300f8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:24.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.862+0000 7f07349ee640 1 -- 172.21.15.102:0/741074280 shutdown_connections 2024-09-11T08:01:24.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.862+0000 7f07349ee640 1 -- 172.21.15.102:0/741074280 wait complete. 2024-09-11T08:01:24.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.863+0000 7f07349ee640 1 Processor -- start 2024-09-11T08:01:24.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.863+0000 7f07349ee640 1 -- start start 2024-09-11T08:01:24.202 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.863+0000 7f07349ee640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f073011bff0 con 0x7f0730105380 2024-09-11T08:01:24.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.863+0000 7f07349ee640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f073011d1b0 con 0x7f0730108ee0 2024-09-11T08:01:24.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.863+0000 7f07349ee640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f073011e3b0 con 0x7f0730102610 2024-09-11T08:01:24.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.863+0000 7f072e59c640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7f0730102610 0x7f07300fe5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:36972/0 (socket says 172.21.15.102:36972) 2024-09-11T08:01:24.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.863+0000 7f072e59c640 1 -- 172.21.15.102:0/2736990799 learned_addr learned my addr 172.21.15.102:0/2736990799 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:01:24.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.863+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035699401 0 0) 0x7f073011bff0 con 0x7f0730105380 2024-09-11T08:01:24.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.863+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0718003200 con 0x7f0730105380 2024-09-11T08:01:24.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.863+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1121872020 0 0) 0x7f073011e3b0 con 0x7f0730102610 2024-09-11T08:01:24.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.863+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f073011bff0 con 0x7f0730102610 2024-09-11T08:01:24.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.863+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3248698674 0 0) 0x7f073011d1b0 con 0x7f0730108ee0 2024-09-11T08:01:24.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.863+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f073011e3b0 con 0x7f0730108ee0 2024-09-11T08:01:24.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.863+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 168379204 0 0) 0x7f0718003200 con 0x7f0730105380 2024-09-11T08:01:24.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.863+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f073011d1b0 con 0x7f0730105380 2024-09-11T08:01:24.203 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.863+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f0724001880 con 0x7f0730105380 2024-09-11T08:01:24.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.863+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 655061418 0 0) 0x7f073011e3b0 con 0x7f0730108ee0 2024-09-11T08:01:24.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.863+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0718003200 con 0x7f0730108ee0 2024-09-11T08:01:24.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.863+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3852128785 0 0) 0x7f073011bff0 con 0x7f0730102610 2024-09-11T08:01:24.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.863+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f073011e3b0 con 0x7f0730102610 2024-09-11T08:01:24.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.863+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f07280019a0 con 0x7f0730108ee0 2024-09-11T08:01:24.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.864+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f0720001430 con 0x7f0730102610 2024-09-11T08:01:24.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.864+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3002680589 0 0) 0x7f073011d1b0 con 0x7f0730105380 2024-09-11T08:01:24.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.864+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 >> v1:172.21.15.102:6790/0 conn(0x7f0730102610 legacy=0x7f07300fe5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:24.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.864+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 >> v1:172.21.15.146:6789/0 conn(0x7f0730108ee0 legacy=0x7f073011a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:24.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.864+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073011f5b0 con 0x7f0730105380 2024-09-11T08:01:24.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.864+0000 7f07349ee640 1 -- 172.21.15.102:0/2736990799 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073011e5a0 con 0x7f0730105380 2024-09-11T08:01:24.204 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.864+0000 7f07349ee640 1 -- 172.21.15.102:0/2736990799 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f073011e9f0 con 0x7f0730105380 2024-09-11T08:01:24.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.864+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f0724001fc0 con 0x7f0730105380 2024-09-11T08:01:24.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.864+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f0724003150 con 0x7f0730105380 2024-09-11T08:01:24.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.865+0000 7f07349ee640 1 -- 172.21.15.102:0/2736990799 --> v1:172.21.15.102:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0710000f20 con 0x7f0730105380 2024-09-11T08:01:24.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.865+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7f0724019630 con 0x7f0730105380 2024-09-11T08:01:24.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.866+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 376332686 0 0) 0x7f0724087640 con 0x7f0730105380 2024-09-11T08:01:24.205 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.866+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 --> v1:172.21.15.146:6800/2908355826 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f071807e260 con 0x7f071807a5c0 2024-09-11T08:01:24.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.866+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (unknown 1962166362 0 0) 0x7f0724087a70 con 0x7f0730105380 2024-09-11T08:01:24.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.867+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 <== osd.4 v1:172.21.15.146:6800/2908355826 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f071807e260 con 0x7f071807a5c0 2024-09-11T08:01:24.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.879+0000 7f07349ee640 1 -- 172.21.15.102:0/2736990799 --> v1:172.21.15.146:6800/2908355826 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0710002b10 con 0x7f071807a5c0 2024-09-11T08:01:24.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.880+0000 7f071effd640 1 -- 172.21.15.102:0/2736990799 <== osd.4 v1:172.21.15.146:6800/2908355826 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 3717429043) 0x7f0710002b10 con 0x7f071807a5c0 2024-09-11T08:01:24.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.880+0000 7f07349ee640 1 -- 172.21.15.102:0/2736990799 >> v1:172.21.15.146:6800/2908355826 conn(0x7f071807a5c0 legacy=0x7f071807c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:24.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.880+0000 7f07349ee640 1 -- 172.21.15.102:0/2736990799 >> v1:172.21.15.102:6800/4281557748 conn(0x7f071806fdb0 legacy=0x7f0718072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:24.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.880+0000 7f07349ee640 1 -- 172.21.15.102:0/2736990799 >> v1:172.21.15.102:6789/0 conn(0x7f0730105380 legacy=0x7f07300fecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:24.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.880+0000 7f072ed9d640 1 -- 172.21.15.102:0/2736990799 reap_dead start 2024-09-11T08:01:24.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.880+0000 7f07349ee640 1 -- 172.21.15.102:0/2736990799 shutdown_connections 2024-09-11T08:01:24.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.880+0000 7f07349ee640 1 -- 172.21.15.102:0/2736990799 >> 172.21.15.102:0/2736990799 conn(0x7f07300749e0 msgr2=0x7f07300f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:24.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.880+0000 7f07349ee640 1 -- 172.21.15.102:0/2736990799 shutdown_connections 2024-09-11T08:01:24.206 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:23.880+0000 7f07349ee640 1 -- 172.21.15.102:0/2736990799 wait complete. 2024-09-11T08:01:24.393 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:01:24.393 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:01:24.393 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:01:24.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:24 smithi102 ceph-mon[29849]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:01:24.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:24 smithi102 ceph-mon[29849]: Updating smithi146:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:01:24.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:24 smithi102 ceph-mon[29849]: Updating smithi102:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:01:24.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:24 smithi102 ceph-mon[29849]: Updating smithi146:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:01:24.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:24 smithi102 ceph-mon[29849]: Deploying daemon alertmanager.a on smithi102 2024-09-11T08:01:24.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:24 smithi102 ceph-mon[41371]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:01:24.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:24 smithi102 ceph-mon[41371]: Updating smithi146:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:01:24.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:24 smithi102 ceph-mon[41371]: Updating smithi102:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:01:24.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:24 smithi102 ceph-mon[41371]: Updating smithi146:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:01:24.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:24 smithi102 ceph-mon[41371]: Deploying daemon alertmanager.a on smithi102 2024-09-11T08:01:24.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:01:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:01:24.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:24 smithi146 ceph-mon[33158]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:01:24.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:24 smithi146 ceph-mon[33158]: Updating smithi146:/etc/ceph/ceph.client.admin.keyring 2024-09-11T08:01:24.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:24 smithi146 ceph-mon[33158]: Updating smithi102:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:01:24.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:24 smithi146 ceph-mon[33158]: Updating smithi146:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T08:01:24.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:24 smithi146 ceph-mon[33158]: Deploying daemon alertmanager.a on smithi102 2024-09-11T08:01:25.115 INFO:teuthology.orchestra.run.smithi102.stdout:55834574890 2024-09-11T08:01:25.116 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-11T08:01:25.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:25 smithi146 ceph-mon[33158]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-11T08:01:25.974 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:25 smithi102 ceph-mon[41371]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-11T08:01:25.975 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:25 smithi102 ceph-mon[29849]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-11T08:01:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:27 smithi146 ceph-mon[33158]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:01:28.045 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:27 smithi102 ceph-mon[41371]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:01:28.046 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:27 smithi102 ceph-mon[29849]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:01:28.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.124+0000 7fa1f2728640 1 -- 172.21.15.102:0/1536887543 >> v1:172.21.15.102:6789/0 conn(0x7fa1ec0fd5b0 legacy=0x7fa1ec0fd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:28.163 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.125+0000 7fa1f2728640 1 -- 172.21.15.102:0/1536887543 shutdown_connections 2024-09-11T08:01:28.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.125+0000 7fa1f2728640 1 -- 172.21.15.102:0/1536887543 >> 172.21.15.102:0/1536887543 conn(0x7fa1ec0752c0 msgr2=0x7fa1ec0f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:28.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.125+0000 7fa1f2728640 1 -- 172.21.15.102:0/1536887543 shutdown_connections 2024-09-11T08:01:28.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.125+0000 7fa1f2728640 1 -- 172.21.15.102:0/1536887543 wait complete. 2024-09-11T08:01:28.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.125+0000 7fa1f2728640 1 Processor -- start 2024-09-11T08:01:28.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.125+0000 7fa1f2728640 1 -- start start 2024-09-11T08:01:28.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.126+0000 7fa1f2728640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa1ec11bfc0 con 0x7fa1ec0fd5b0 2024-09-11T08:01:28.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.126+0000 7fa1f2728640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa1ec11d1c0 con 0x7fa1ec108f10 2024-09-11T08:01:28.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.126+0000 7fa1f2728640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa1ec11e3c0 con 0x7fa1ec0fe180 2024-09-11T08:01:28.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.126+0000 7fa1ebfff640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fa1ec0fd5b0 0x7fa1ec104210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:54820/0 (socket says 172.21.15.102:54820) 2024-09-11T08:01:28.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.126+0000 7fa1ebfff640 1 -- 172.21.15.102:0/3853421984 learned_addr learned my addr 172.21.15.102:0/3853421984 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:01:28.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.126+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1730138753 0 0) 0x7fa1ec11e3c0 con 0x7fa1ec0fe180 2024-09-11T08:01:28.164 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.126+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1d4003200 con 0x7fa1ec0fe180 2024-09-11T08:01:28.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.126+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 660300752 0 0) 0x7fa1ec11bfc0 con 0x7fa1ec0fd5b0 2024-09-11T08:01:28.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.126+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1ec11e3c0 con 0x7fa1ec0fd5b0 2024-09-11T08:01:28.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.126+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3065471715 0 0) 0x7fa1ec11d1c0 con 0x7fa1ec108f10 2024-09-11T08:01:28.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.126+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1ec11bfc0 con 0x7fa1ec108f10 2024-09-11T08:01:28.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.126+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1074467139 0 0) 0x7fa1d4003200 con 0x7fa1ec0fe180 2024-09-11T08:01:28.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.126+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa1ec11d1c0 con 0x7fa1ec0fe180 2024-09-11T08:01:28.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.126+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fa1e0001880 con 0x7fa1ec0fe180 2024-09-11T08:01:28.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.126+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 818761430 0 0) 0x7fa1ec11e3c0 con 0x7fa1ec0fd5b0 2024-09-11T08:01:28.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.126+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa1d4003200 con 0x7fa1ec0fd5b0 2024-09-11T08:01:28.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.127+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fa1dc001500 con 0x7fa1ec0fd5b0 2024-09-11T08:01:28.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.127+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 260352596 0 0) 0x7fa1ec11bfc0 con 0x7fa1ec108f10 2024-09-11T08:01:28.165 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.127+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa1ec11e3c0 con 0x7fa1ec108f10 2024-09-11T08:01:28.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.127+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3060737218 0 0) 0x7fa1ec11d1c0 con 0x7fa1ec0fe180 2024-09-11T08:01:28.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.127+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 >> v1:172.21.15.146:6789/0 conn(0x7fa1ec108f10 legacy=0x7fa1ec11a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:28.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.127+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 >> v1:172.21.15.102:6789/0 conn(0x7fa1ec0fd5b0 legacy=0x7fa1ec104210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:28.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.127+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec11f5c0 con 0x7fa1ec0fe180 2024-09-11T08:01:28.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.127+0000 7fa1f2728640 1 -- 172.21.15.102:0/3853421984 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1ec11d3b0 con 0x7fa1ec0fe180 2024-09-11T08:01:28.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.127+0000 7fa1f2728640 1 -- 172.21.15.102:0/3853421984 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ec11d800 con 0x7fa1ec0fe180 2024-09-11T08:01:28.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.127+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7fa1e0001ef0 con 0x7fa1ec0fe180 2024-09-11T08:01:28.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.127+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fa1e0003150 con 0x7fa1ec0fe180 2024-09-11T08:01:28.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.128+0000 7fa1f2728640 1 -- 172.21.15.102:0/3853421984 --> v1:172.21.15.102:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa1d8000f20 con 0x7fa1ec0fe180 2024-09-11T08:01:28.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.129+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7fa1e0019570 con 0x7fa1ec0fe180 2024-09-11T08:01:28.166 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.129+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 376332686 0 0) 0x7fa1e00874f0 con 0x7fa1ec0fe180 2024-09-11T08:01:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.129+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 --> v1:172.21.15.102:6809/3842496257 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa1d407e400 con 0x7fa1d407a760 2024-09-11T08:01:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.129+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (unknown 1962166362 0 0) 0x7fa1e00878c0 con 0x7fa1ec0fe180 2024-09-11T08:01:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.130+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 <== osd.2 v1:172.21.15.102:6809/3842496257 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7fa1d407e400 con 0x7fa1d407a760 2024-09-11T08:01:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.147+0000 7fa1f2728640 1 -- 172.21.15.102:0/3853421984 --> v1:172.21.15.102:6809/3842496257 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa1d8002b10 con 0x7fa1d407a760 2024-09-11T08:01:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.147+0000 7fa1e8ff9640 1 -- 172.21.15.102:0/3853421984 <== osd.2 v1:172.21.15.102:6809/3842496257 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2431347361) 0x7fa1d8002b10 con 0x7fa1d407a760 2024-09-11T08:01:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.148+0000 7fa1f2728640 1 -- 172.21.15.102:0/3853421984 >> v1:172.21.15.102:6809/3842496257 conn(0x7fa1d407a760 legacy=0x7fa1d407cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.148+0000 7fa1f2728640 1 -- 172.21.15.102:0/3853421984 >> v1:172.21.15.102:6800/4281557748 conn(0x7fa1d406fe50 legacy=0x7fa1d4072300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.148+0000 7fa1f2728640 1 -- 172.21.15.102:0/3853421984 >> v1:172.21.15.102:6790/0 conn(0x7fa1ec0fe180 legacy=0x7fa1ec104920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.148+0000 7fa1f0cc5640 1 -- 172.21.15.102:0/3853421984 reap_dead start 2024-09-11T08:01:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.148+0000 7fa1f2728640 1 -- 172.21.15.102:0/3853421984 shutdown_connections 2024-09-11T08:01:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.148+0000 7fa1f2728640 1 -- 172.21.15.102:0/3853421984 >> 172.21.15.102:0/3853421984 conn(0x7fa1ec0752c0 msgr2=0x7fa1ec0f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.148+0000 7fa1f2728640 1 -- 172.21.15.102:0/3853421984 shutdown_connections 2024-09-11T08:01:28.167 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:28.148+0000 7fa1f2728640 1 -- 172.21.15.102:0/3853421984 wait complete. 2024-09-11T08:01:29.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:01:29 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:01:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:29 smithi146 ceph-mon[33158]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:01:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:29 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:01:29.957 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:29 smithi102 ceph-mon[41371]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:01:29.957 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:29 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:01:29.958 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:29 smithi102 ceph-mon[29849]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:01:29.958 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:29 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:01:30.168 INFO:teuthology.orchestra.run.smithi102.stdout:77309411366 2024-09-11T08:01:30.168 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-11T08:01:30.801 INFO:teuthology.orchestra.run.smithi102.stdout:128849018908 2024-09-11T08:01:30.801 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-11T08:01:31.004 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:30 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:01:31.005 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:30 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:01:31.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:30 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:01:31.422 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:01:31.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:31 smithi102 ceph-mon[29849]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:01:31.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:31 smithi102 ceph-mon[41371]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:01:32.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:31 smithi146 ceph-mon[33158]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T08:01:32.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.662+0000 7f1038d31640 1 -- 172.21.15.102:0/2760705908 >> v1:172.21.15.102:6790/0 conn(0x7f1034101710 legacy=0x7f1034101ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:32.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.662+0000 7f1038d31640 1 -- 172.21.15.102:0/2760705908 shutdown_connections 2024-09-11T08:01:32.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.662+0000 7f1038d31640 1 -- 172.21.15.102:0/2760705908 >> 172.21.15.102:0/2760705908 conn(0x7f1034073690 msgr2=0x7f1034073a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:32.667 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.662+0000 7f1038d31640 1 -- 172.21.15.102:0/2760705908 shutdown_connections 2024-09-11T08:01:32.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.662+0000 7f1038d31640 1 -- 172.21.15.102:0/2760705908 wait complete. 2024-09-11T08:01:32.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.662+0000 7f1038d31640 1 Processor -- start 2024-09-11T08:01:32.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.662+0000 7f1038d31640 1 -- start start 2024-09-11T08:01:32.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f1038d31640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f103411c150 con 0x7f10341044a0 2024-09-11T08:01:32.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f1038d31640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f103411d310 con 0x7f1034108030 2024-09-11T08:01:32.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f1038d31640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f103411e510 con 0x7f1034101710 2024-09-11T08:01:32.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f103259c640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7f1034101710 0x7f1034065d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:40432/0 (socket says 172.21.15.102:40432) 2024-09-11T08:01:32.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f103259c640 1 -- 172.21.15.102:0/70454426 learned_addr learned my addr 172.21.15.102:0/70454426 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:01:32.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2346231276 0 0) 0x7f103411e510 con 0x7f1034101710 2024-09-11T08:01:32.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f101c0034e0 con 0x7f1034101710 2024-09-11T08:01:32.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028171751 0 0) 0x7f103411c150 con 0x7f10341044a0 2024-09-11T08:01:32.668 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f103411e510 con 0x7f10341044a0 2024-09-11T08:01:32.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 820573988 0 0) 0x7f103411d310 con 0x7f1034108030 2024-09-11T08:01:32.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f103411c150 con 0x7f1034108030 2024-09-11T08:01:32.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2137910454 0 0) 0x7f103411e510 con 0x7f10341044a0 2024-09-11T08:01:32.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f103411d310 con 0x7f10341044a0 2024-09-11T08:01:32.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f10280019f0 con 0x7f10341044a0 2024-09-11T08:01:32.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3996605436 0 0) 0x7f101c0034e0 con 0x7f1034101710 2024-09-11T08:01:32.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f103411e510 con 0x7f1034101710 2024-09-11T08:01:32.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f10240029e0 con 0x7f1034101710 2024-09-11T08:01:32.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3180852926 0 0) 0x7f103411d310 con 0x7f10341044a0 2024-09-11T08:01:32.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 >> v1:172.21.15.102:6790/0 conn(0x7f1034101710 legacy=0x7f1034065d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:32.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 >> v1:172.21.15.146:6789/0 conn(0x7f1034108030 legacy=0x7f103411a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:32.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103411f710 con 0x7f10341044a0 2024-09-11T08:01:32.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f1038d31640 1 -- 172.21.15.102:0/70454426 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f103411c340 con 0x7f10341044a0 2024-09-11T08:01:32.669 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f1038d31640 1 -- 172.21.15.102:0/70454426 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f103411c810 con 0x7f10341044a0 2024-09-11T08:01:32.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f1028002310 con 0x7f10341044a0 2024-09-11T08:01:32.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.663+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f10280031d0 con 0x7f10341044a0 2024-09-11T08:01:32.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.664+0000 7f1038d31640 1 -- 172.21.15.102:0/70454426 --> v1:172.21.15.102:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1014000f20 con 0x7f10341044a0 2024-09-11T08:01:32.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.665+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7f1028001ec0 con 0x7f10341044a0 2024-09-11T08:01:32.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.665+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 376332686 0 0) 0x7f1028087460 con 0x7f10341044a0 2024-09-11T08:01:32.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.665+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 --> v1:172.21.15.102:6813/426874071 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f101c07e0a0 con 0x7f101c07a400 2024-09-11T08:01:32.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.665+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (unknown 1962166362 0 0) 0x7f1028087830 con 0x7f10341044a0 2024-09-11T08:01:32.670 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.666+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 <== osd.3 v1:172.21.15.102:6813/426874071 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f101c07e0a0 con 0x7f101c07a400 2024-09-11T08:01:32.680 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.679+0000 7f1038d31640 1 -- 172.21.15.102:0/70454426 --> v1:172.21.15.102:6813/426874071 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1014002b10 con 0x7f101c07a400 2024-09-11T08:01:32.681 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.680+0000 7f1022ffd640 1 -- 172.21.15.102:0/70454426 <== osd.3 v1:172.21.15.102:6813/426874071 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 325371873) 0x7f1014002b10 con 0x7f101c07a400 2024-09-11T08:01:32.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.680+0000 7f1038d31640 1 -- 172.21.15.102:0/70454426 >> v1:172.21.15.102:6813/426874071 conn(0x7f101c07a400 legacy=0x7f101c07c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:32.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.680+0000 7f1038d31640 1 -- 172.21.15.102:0/70454426 >> v1:172.21.15.102:6800/4281557748 conn(0x7f101c06fb80 legacy=0x7f101c072030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:32.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.681+0000 7f1038d31640 1 -- 172.21.15.102:0/70454426 >> v1:172.21.15.102:6789/0 conn(0x7f10341044a0 legacy=0x7f1034066460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:32.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.681+0000 7f1032d9d640 1 -- 172.21.15.102:0/70454426 reap_dead start 2024-09-11T08:01:32.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.681+0000 7f1038d31640 1 -- 172.21.15.102:0/70454426 shutdown_connections 2024-09-11T08:01:32.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.681+0000 7f1038d31640 1 -- 172.21.15.102:0/70454426 >> 172.21.15.102:0/70454426 conn(0x7f1034073690 msgr2=0x7f1034073e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:32.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.681+0000 7f1038d31640 1 -- 172.21.15.102:0/70454426 shutdown_connections 2024-09-11T08:01:32.682 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:32.681+0000 7f1038d31640 1 -- 172.21.15.102:0/70454426 wait complete. 2024-09-11T08:01:33.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.744+0000 7f05a4c37640 1 -- 172.21.15.102:0/2993240674 >> v1:172.21.15.102:6789/0 conn(0x7f05a0107df0 legacy=0x7f05a010a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:33.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.744+0000 7f05a4c37640 1 -- 172.21.15.102:0/2993240674 shutdown_connections 2024-09-11T08:01:33.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.744+0000 7f05a4c37640 1 -- 172.21.15.102:0/2993240674 >> 172.21.15.102:0/2993240674 conn(0x7f05a00f70a0 msgr2=0x7f05a00f94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:33.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.744+0000 7f05a4c37640 1 -- 172.21.15.102:0/2993240674 shutdown_connections 2024-09-11T08:01:33.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.744+0000 7f05a4c37640 1 -- 172.21.15.102:0/2993240674 wait complete. 2024-09-11T08:01:33.766 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.744+0000 7f05a4c37640 1 Processor -- start 2024-09-11T08:01:33.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.744+0000 7f05a4c37640 1 -- start start 2024-09-11T08:01:33.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.745+0000 7f05a4c37640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f05a011bf20 con 0x7f05a0107df0 2024-09-11T08:01:33.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.745+0000 7f05a4c37640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f05a011d120 con 0x7f05a0104290 2024-09-11T08:01:33.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.745+0000 7f05a4c37640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f05a011e320 con 0x7f05a0101520 2024-09-11T08:01:33.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.745+0000 7f059e59c640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7f05a0101520 0x7f05a0105250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:40448/0 (socket says 172.21.15.102:40448) 2024-09-11T08:01:33.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.745+0000 7f059e59c640 1 -- 172.21.15.102:0/3855513260 learned_addr learned my addr 172.21.15.102:0/3855513260 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:01:33.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.745+0000 7f059ed9d640 1 --1- 172.21.15.102:0/3855513260 >> v1:172.21.15.102:6789/0 conn(0x7f05a0107df0 0x7f05a011a620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:54858/0 (socket says 172.21.15.102:54858) 2024-09-11T08:01:33.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.745+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2004080352 0 0) 0x7f05a011e320 con 0x7f05a0101520 2024-09-11T08:01:33.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.745+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05880034e0 con 0x7f05a0101520 2024-09-11T08:01:33.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.745+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1073931745 0 0) 0x7f05a011bf20 con 0x7f05a0107df0 2024-09-11T08:01:33.767 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.745+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05a011e320 con 0x7f05a0107df0 2024-09-11T08:01:33.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.745+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1292134197 0 0) 0x7f05a011d120 con 0x7f05a0104290 2024-09-11T08:01:33.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.745+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05a011bf20 con 0x7f05a0104290 2024-09-11T08:01:33.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.745+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2391918658 0 0) 0x7f05a011e320 con 0x7f05a0107df0 2024-09-11T08:01:33.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.745+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f05a011d120 con 0x7f05a0107df0 2024-09-11T08:01:33.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.745+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f0598001420 con 0x7f05a0107df0 2024-09-11T08:01:33.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.745+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3284243700 0 0) 0x7f05880034e0 con 0x7f05a0101520 2024-09-11T08:01:33.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.745+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f05a011e320 con 0x7f05a0101520 2024-09-11T08:01:33.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.745+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f0590001980 con 0x7f05a0101520 2024-09-11T08:01:33.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.745+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1825223838 0 0) 0x7f05a011bf20 con 0x7f05a0104290 2024-09-11T08:01:33.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.745+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f05880034e0 con 0x7f05a0104290 2024-09-11T08:01:33.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.745+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f05940018c0 con 0x7f05a0104290 2024-09-11T08:01:33.768 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.746+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1927557054 0 0) 0x7f05a011d120 con 0x7f05a0107df0 2024-09-11T08:01:33.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.746+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 >> v1:172.21.15.102:6790/0 conn(0x7f05a0101520 legacy=0x7f05a0105250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:33.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.746+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 >> v1:172.21.15.146:6789/0 conn(0x7f05a0104290 legacy=0x7f05a0105960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:33.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.746+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a011f520 con 0x7f05a0107df0 2024-09-11T08:01:33.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.746+0000 7f05a4c37640 1 -- 172.21.15.102:0/3855513260 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a011c110 con 0x7f05a0107df0 2024-09-11T08:01:33.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.746+0000 7f05a4c37640 1 -- 172.21.15.102:0/3855513260 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a011c560 con 0x7f05a0107df0 2024-09-11T08:01:33.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.746+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f0598002510 con 0x7f05a0107df0 2024-09-11T08:01:33.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.746+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f0598003780 con 0x7f05a0107df0 2024-09-11T08:01:33.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.746+0000 7f05a4c37640 1 -- 172.21.15.102:0/3855513260 --> v1:172.21.15.102:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0580000f20 con 0x7f05a0107df0 2024-09-11T08:01:33.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.747+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7f0598001b80 con 0x7f05a0107df0 2024-09-11T08:01:33.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.748+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 376332686 0 0) 0x7f059808a2a0 con 0x7f05a0107df0 2024-09-11T08:01:33.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.748+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 --> v1:172.21.15.146:6808/85686736 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f058807e420 con 0x7f058807a780 2024-09-11T08:01:33.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.748+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (unknown 1962166362 0 0) 0x7f059808a6d0 con 0x7f05a0107df0 2024-09-11T08:01:33.769 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.749+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 <== osd.6 v1:172.21.15.146:6808/85686736 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f058807e420 con 0x7f058807a780 2024-09-11T08:01:33.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.761+0000 7f05a4c37640 1 -- 172.21.15.102:0/3855513260 --> v1:172.21.15.146:6808/85686736 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0580002b10 con 0x7f058807a780 2024-09-11T08:01:33.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.762+0000 7f058effd640 1 -- 172.21.15.102:0/3855513260 <== osd.6 v1:172.21.15.146:6808/85686736 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 2208188497) 0x7f0580002b10 con 0x7f058807a780 2024-09-11T08:01:33.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.762+0000 7f05a4c37640 1 -- 172.21.15.102:0/3855513260 >> v1:172.21.15.146:6808/85686736 conn(0x7f058807a780 legacy=0x7f058807cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:33.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.762+0000 7f05a4c37640 1 -- 172.21.15.102:0/3855513260 >> v1:172.21.15.102:6800/4281557748 conn(0x7f058806ff00 legacy=0x7f05880723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:33.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.762+0000 7f05a4c37640 1 -- 172.21.15.102:0/3855513260 >> v1:172.21.15.102:6789/0 conn(0x7f05a0107df0 legacy=0x7f05a011a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:33.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.762+0000 7f059ed9d640 1 -- 172.21.15.102:0/3855513260 reap_dead start 2024-09-11T08:01:33.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.762+0000 7f05a4c37640 1 -- 172.21.15.102:0/3855513260 shutdown_connections 2024-09-11T08:01:33.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.762+0000 7f05a4c37640 1 -- 172.21.15.102:0/3855513260 >> 172.21.15.102:0/3855513260 conn(0x7f05a00f70a0 msgr2=0x7f05a0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:33.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.762+0000 7f05a4c37640 1 -- 172.21.15.102:0/3855513260 shutdown_connections 2024-09-11T08:01:33.770 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:33.762+0000 7f05a4c37640 1 -- 172.21.15.102:0/3855513260 wait complete. 2024-09-11T08:01:34.044 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:33 smithi102 ceph-mon[41371]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:01:34.044 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:33 smithi102 ceph-mon[29849]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:01:34.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:33 smithi146 ceph-mon[33158]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T08:01:34.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:34 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:01:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:01:35.954 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:35 smithi146 ceph-mon[33158]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:35.999 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:35 smithi102 ceph-mon[29849]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:36.000 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:35 smithi102 ceph-mon[41371]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:36.036 INFO:teuthology.orchestra.run.smithi102.stdout:107374182434 2024-09-11T08:01:36.036 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-11T08:01:36.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.854+0000 7fcdc3706640 1 -- 172.21.15.102:0/803989556 >> v1:172.21.15.102:6789/0 conn(0x7fcdbc101530 legacy=0x7fcdbc101900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:36.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.854+0000 7fcdc3706640 1 -- 172.21.15.102:0/803989556 shutdown_connections 2024-09-11T08:01:36.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.854+0000 7fcdc3706640 1 -- 172.21.15.102:0/803989556 >> 172.21.15.102:0/803989556 conn(0x7fcdbc0752c0 msgr2=0x7fcdbc0f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:36.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.854+0000 7fcdc3706640 1 -- 172.21.15.102:0/803989556 shutdown_connections 2024-09-11T08:01:36.856 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.854+0000 7fcdc3706640 1 -- 172.21.15.102:0/803989556 wait complete. 2024-09-11T08:01:36.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.855+0000 7fcdc3706640 1 Processor -- start 2024-09-11T08:01:36.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.855+0000 7fcdc3706640 1 -- start start 2024-09-11T08:01:36.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.855+0000 7fcdc3706640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcdbc122750 con 0x7fcdbc101530 2024-09-11T08:01:36.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.855+0000 7fcdc3706640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcdbc1238f0 con 0x7fcdbc107e00 2024-09-11T08:01:36.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.855+0000 7fcdc3706640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcdbc124a90 con 0x7fcdbc1042a0 2024-09-11T08:01:36.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.855+0000 7fcdc0ca1640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7fcdbc1042a0 0x7fcdbc065720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:40462/0 (socket says 172.21.15.102:40462) 2024-09-11T08:01:36.858 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.855+0000 7fcdc0ca1640 1 -- 172.21.15.102:0/3607670574 learned_addr learned my addr 172.21.15.102:0/3607670574 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:01:36.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.856+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 65435131 0 0) 0x7fcdbc124a90 con 0x7fcdbc1042a0 2024-09-11T08:01:36.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.856+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcda40034e0 con 0x7fcdbc1042a0 2024-09-11T08:01:36.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.856+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 203695386 0 0) 0x7fcdbc1238f0 con 0x7fcdbc107e00 2024-09-11T08:01:36.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.856+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdbc124a90 con 0x7fcdbc107e00 2024-09-11T08:01:36.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.856+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636684066 0 0) 0x7fcdbc122750 con 0x7fcdbc101530 2024-09-11T08:01:36.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.856+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdbc1238f0 con 0x7fcdbc101530 2024-09-11T08:01:36.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.856+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1620014300 0 0) 0x7fcdbc124a90 con 0x7fcdbc107e00 2024-09-11T08:01:36.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.856+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcdbc122750 con 0x7fcdbc107e00 2024-09-11T08:01:36.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.856+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fcdb40018c0 con 0x7fcdbc107e00 2024-09-11T08:01:36.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.856+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2799059733 0 0) 0x7fcda40034e0 con 0x7fcdbc1042a0 2024-09-11T08:01:36.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.856+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcdbc124a90 con 0x7fcdbc1042a0 2024-09-11T08:01:36.859 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.856+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2739226153 0 0) 0x7fcdbc1238f0 con 0x7fcdbc101530 2024-09-11T08:01:36.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.856+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcda40034e0 con 0x7fcdbc101530 2024-09-11T08:01:36.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.856+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fcdb00028d0 con 0x7fcdbc1042a0 2024-09-11T08:01:36.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.856+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fcdac002f30 con 0x7fcdbc101530 2024-09-11T08:01:36.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.856+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 <== mon.1 v1:172.21.15.146:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3969517103 0 0) 0x7fcdbc122750 con 0x7fcdbc107e00 2024-09-11T08:01:36.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.856+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 >> v1:172.21.15.102:6790/0 conn(0x7fcdbc1042a0 legacy=0x7fcdbc065720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:36.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.856+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 >> v1:172.21.15.102:6789/0 conn(0x7fcdbc101530 legacy=0x7fcdbc065010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:36.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.856+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdbc125c30 con 0x7fcdbc107e00 2024-09-11T08:01:36.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.856+0000 7fcdc3706640 1 -- 172.21.15.102:0/3607670574 --> v1:172.21.15.146:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdbc123ae0 con 0x7fcdbc107e00 2024-09-11T08:01:36.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.856+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 <== mon.1 v1:172.21.15.146:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7fcdb4002040 con 0x7fcdbc107e00 2024-09-11T08:01:36.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.856+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 <== mon.1 v1:172.21.15.146:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fcdb40031c0 con 0x7fcdbc107e00 2024-09-11T08:01:36.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.857+0000 7fcdc3706640 1 -- 172.21.15.102:0/3607670574 --> v1:172.21.15.146:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdbc123fb0 con 0x7fcdbc107e00 2024-09-11T08:01:36.860 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.857+0000 7fcdc3706640 1 -- 172.21.15.102:0/3607670574 --> v1:172.21.15.146:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcdbc1244e0 con 0x7fcdbc107e00 2024-09-11T08:01:36.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.858+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 <== mon.1 v1:172.21.15.146:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7fcdb4019710 con 0x7fcdbc107e00 2024-09-11T08:01:36.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.858+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 <== mon.1 v1:172.21.15.146:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 376332686 0 0) 0x7fcdb40876f0 con 0x7fcdbc107e00 2024-09-11T08:01:36.861 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.858+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 --> v1:172.21.15.146:6804/1497668584 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcda407e400 con 0x7fcda407a760 2024-09-11T08:01:36.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.859+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 <== mon.1 v1:172.21.15.146:6789/0 9 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (unknown 1962166362 0 0) 0x7fcdb4087ac0 con 0x7fcdbc107e00 2024-09-11T08:01:36.862 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.860+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 <== osd.5 v1:172.21.15.146:6804/1497668584 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7fcda407e400 con 0x7fcda407a760 2024-09-11T08:01:36.878 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.877+0000 7fcdc3706640 1 -- 172.21.15.102:0/3607670574 --> v1:172.21.15.146:6804/1497668584 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcdbc124890 con 0x7fcda407a760 2024-09-11T08:01:36.879 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.878+0000 7fcdb9ffb640 1 -- 172.21.15.102:0/3607670574 <== osd.5 v1:172.21.15.146:6804/1497668584 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 4032845322) 0x7fcdbc124890 con 0x7fcda407a760 2024-09-11T08:01:36.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.878+0000 7fcdc3706640 1 -- 172.21.15.102:0/3607670574 >> v1:172.21.15.146:6804/1497668584 conn(0x7fcda407a760 legacy=0x7fcda407cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:36.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.878+0000 7fcdc3706640 1 -- 172.21.15.102:0/3607670574 >> v1:172.21.15.102:6800/4281557748 conn(0x7fcda406fdd0 legacy=0x7fcda4072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:36.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.878+0000 7fcdc3706640 1 -- 172.21.15.102:0/3607670574 >> v1:172.21.15.146:6789/0 conn(0x7fcdbc107e00 legacy=0x7fcdbc121030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:36.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.878+0000 7fcdc1ca3640 1 -- 172.21.15.102:0/3607670574 reap_dead start 2024-09-11T08:01:36.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.878+0000 7fcdc3706640 1 -- 172.21.15.102:0/3607670574 shutdown_connections 2024-09-11T08:01:36.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.878+0000 7fcdc3706640 1 -- 172.21.15.102:0/3607670574 >> 172.21.15.102:0/3607670574 conn(0x7fcdbc0752c0 msgr2=0x7fcdbc0f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:36.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.878+0000 7fcdc3706640 1 -- 172.21.15.102:0/3607670574 shutdown_connections 2024-09-11T08:01:36.880 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:36.878+0000 7fcdc3706640 1 -- 172.21.15.102:0/3607670574 wait complete. 2024-09-11T08:01:38.057 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:37 smithi102 ceph-mon[41371]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2024-09-11T08:01:38.058 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:37 smithi102 ceph-mon[29849]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2024-09-11T08:01:38.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:37 smithi146 ceph-mon[33158]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2024-09-11T08:01:38.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.353+0000 7fb7525f7640 1 -- 172.21.15.102:0/2655843443 >> v1:172.21.15.102:6789/0 conn(0x7fb74c0fe1a0 legacy=0x7fb74c107650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:38.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.353+0000 7fb7525f7640 1 -- 172.21.15.102:0/2655843443 shutdown_connections 2024-09-11T08:01:38.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.353+0000 7fb7525f7640 1 -- 172.21.15.102:0/2655843443 >> 172.21.15.102:0/2655843443 conn(0x7fb74c0752c0 msgr2=0x7fb74c0f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:38.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.353+0000 7fb7525f7640 1 -- 172.21.15.102:0/2655843443 shutdown_connections 2024-09-11T08:01:38.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.353+0000 7fb7525f7640 1 -- 172.21.15.102:0/2655843443 wait complete. 2024-09-11T08:01:38.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.354+0000 7fb7525f7640 1 Processor -- start 2024-09-11T08:01:38.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.354+0000 7fb7525f7640 1 -- start start 2024-09-11T08:01:38.385 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.355+0000 7fb7525f7640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb74c11bfb0 con 0x7fb74c0fd5d0 2024-09-11T08:01:38.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.355+0000 7fb7525f7640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb74c11d1b0 con 0x7fb74c0fe1a0 2024-09-11T08:01:38.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.355+0000 7fb7525f7640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb74c11e3b0 con 0x7fb74c108f00 2024-09-11T08:01:38.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.355+0000 7fb74bfff640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb74c0fd5d0 0x7fb74c104230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:40990/0 (socket says 172.21.15.102:40990) 2024-09-11T08:01:38.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.355+0000 7fb750b94640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7fb74c108f00 0x7fb74c11a6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:36002/0 (socket says 172.21.15.102:36002) 2024-09-11T08:01:38.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.355+0000 7fb74bfff640 1 -- 172.21.15.102:0/2439375686 learned_addr learned my addr 172.21.15.102:0/2439375686 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:01:38.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.355+0000 7fb748ff9640 1 -- 172.21.15.102:0/2439375686 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808517389 0 0) 0x7fb74c11bfb0 con 0x7fb74c0fd5d0 2024-09-11T08:01:38.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.355+0000 7fb748ff9640 1 -- 172.21.15.102:0/2439375686 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7380034e0 con 0x7fb74c0fd5d0 2024-09-11T08:01:38.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.355+0000 7fb748ff9640 1 -- 172.21.15.102:0/2439375686 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3450099294 0 0) 0x7fb74c11e3b0 con 0x7fb74c108f00 2024-09-11T08:01:38.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.355+0000 7fb748ff9640 1 -- 172.21.15.102:0/2439375686 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb74c11bfb0 con 0x7fb74c108f00 2024-09-11T08:01:38.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.355+0000 7fb748ff9640 1 -- 172.21.15.102:0/2439375686 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2783284032 0 0) 0x7fb74c11bfb0 con 0x7fb74c108f00 2024-09-11T08:01:38.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.355+0000 7fb748ff9640 1 -- 172.21.15.102:0/2439375686 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb74c11e3b0 con 0x7fb74c108f00 2024-09-11T08:01:38.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.355+0000 7fb748ff9640 1 -- 172.21.15.102:0/2439375686 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fb744001cf0 con 0x7fb74c108f00 2024-09-11T08:01:38.386 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.355+0000 7fb748ff9640 1 -- 172.21.15.102:0/2439375686 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1402025425 0 0) 0x7fb7380034e0 con 0x7fb74c0fd5d0 2024-09-11T08:01:38.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.355+0000 7fb748ff9640 1 -- 172.21.15.102:0/2439375686 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb74c11bfb0 con 0x7fb74c0fd5d0 2024-09-11T08:01:38.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.355+0000 7fb748ff9640 1 -- 172.21.15.102:0/2439375686 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1934648972 0 0) 0x7fb74c11d1b0 con 0x7fb74c0fe1a0 2024-09-11T08:01:38.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.355+0000 7fb748ff9640 1 -- 172.21.15.102:0/2439375686 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7380034e0 con 0x7fb74c0fe1a0 2024-09-11T08:01:38.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.355+0000 7fb748ff9640 1 -- 172.21.15.102:0/2439375686 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fb7340018c0 con 0x7fb74c0fd5d0 2024-09-11T08:01:38.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.356+0000 7fb748ff9640 1 -- 172.21.15.102:0/2439375686 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3723792563 0 0) 0x7fb74c11e3b0 con 0x7fb74c108f00 2024-09-11T08:01:38.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.356+0000 7fb748ff9640 1 -- 172.21.15.102:0/2439375686 >> v1:172.21.15.146:6789/0 conn(0x7fb74c0fe1a0 legacy=0x7fb74c104940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:38.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.356+0000 7fb748ff9640 1 -- 172.21.15.102:0/2439375686 >> v1:172.21.15.102:6789/0 conn(0x7fb74c0fd5d0 legacy=0x7fb74c104230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:38.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.356+0000 7fb748ff9640 1 -- 172.21.15.102:0/2439375686 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb74c11f5b0 con 0x7fb74c108f00 2024-09-11T08:01:38.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.356+0000 7fb7525f7640 1 -- 172.21.15.102:0/2439375686 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb74c11c1a0 con 0x7fb74c108f00 2024-09-11T08:01:38.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.356+0000 7fb7525f7640 1 -- 172.21.15.102:0/2439375686 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb74c11c5f0 con 0x7fb74c108f00 2024-09-11T08:01:38.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.356+0000 7fb748ff9640 1 -- 172.21.15.102:0/2439375686 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7fb744001900 con 0x7fb74c108f00 2024-09-11T08:01:38.387 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.356+0000 7fb748ff9640 1 -- 172.21.15.102:0/2439375686 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fb744005b80 con 0x7fb74c108f00 2024-09-11T08:01:38.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.357+0000 7fb7525f7640 1 -- 172.21.15.102:0/2439375686 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72c005010 con 0x7fb74c108f00 2024-09-11T08:01:38.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.357+0000 7fb748ff9640 1 -- 172.21.15.102:0/2439375686 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7fb74401bfa0 con 0x7fb74c108f00 2024-09-11T08:01:38.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.358+0000 7fb748ff9640 1 -- 172.21.15.102:0/2439375686 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 376332686 0 0) 0x7fb744089f40 con 0x7fb74c108f00 2024-09-11T08:01:38.388 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.362+0000 7fb748ff9640 1 -- 172.21.15.102:0/2439375686 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fb7440567b0 con 0x7fb74c108f00 2024-09-11T08:01:38.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.534+0000 7fb7525f7640 1 -- 172.21.15.102:0/2439375686 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb72c0052c0 con 0x7fb74c108f00 2024-09-11T08:01:38.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.535+0000 7fb748ff9640 1 -- 172.21.15.102:0/2439375686 <== mon.2 v1:172.21.15.102:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (unknown 574334944 0 2293705356) 0x7fb7440565d0 con 0x7fb74c108f00 2024-09-11T08:01:38.537 INFO:teuthology.orchestra.run.smithi102.stdout:34359738417 2024-09-11T08:01:38.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.537+0000 7fb7525f7640 1 -- 172.21.15.102:0/2439375686 >> v1:172.21.15.102:6800/4281557748 conn(0x7fb73806ff60 legacy=0x7fb738072410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:38.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.537+0000 7fb7525f7640 1 -- 172.21.15.102:0/2439375686 >> v1:172.21.15.102:6790/0 conn(0x7fb74c108f00 legacy=0x7fb74c11a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:38.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.537+0000 7fb7525f7640 1 -- 172.21.15.102:0/2439375686 shutdown_connections 2024-09-11T08:01:38.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.537+0000 7fb7525f7640 1 -- 172.21.15.102:0/2439375686 >> 172.21.15.102:0/2439375686 conn(0x7fb74c0752c0 msgr2=0x7fb74c0f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:38.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.537+0000 7fb7525f7640 1 -- 172.21.15.102:0/2439375686 shutdown_connections 2024-09-11T08:01:38.538 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:38.537+0000 7fb7525f7640 1 -- 172.21.15.102:0/2439375686 wait complete. 2024-09-11T08:01:38.646 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:01:38 smithi102 systemd[1]: Starting Ceph alertmanager.a for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:01:39.005 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2439375686' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T08:01:39.005 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2439375686' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T08:01:39.030 INFO:teuthology.orchestra.run.smithi102.stdout:154618822683 2024-09-11T08:01:39.030 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-11T08:01:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2439375686' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T08:01:39.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.511+0000 7f6601409640 1 -- 172.21.15.102:0/72747040 >> v1:172.21.15.102:6789/0 conn(0x7f65fc101530 legacy=0x7f65fc101900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:39.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.511+0000 7f6601409640 1 -- 172.21.15.102:0/72747040 shutdown_connections 2024-09-11T08:01:39.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.511+0000 7f6601409640 1 -- 172.21.15.102:0/72747040 >> 172.21.15.102:0/72747040 conn(0x7f65fc0752c0 msgr2=0x7f65fc0f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:39.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.511+0000 7f6601409640 1 -- 172.21.15.102:0/72747040 shutdown_connections 2024-09-11T08:01:39.514 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.511+0000 7f6601409640 1 -- 172.21.15.102:0/72747040 wait complete. 2024-09-11T08:01:39.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.511+0000 7f6601409640 1 Processor -- start 2024-09-11T08:01:39.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.511+0000 7f6601409640 1 -- start start 2024-09-11T08:01:39.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.511+0000 7f6601409640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f65fc11ac90 con 0x7f65fc107e00 2024-09-11T08:01:39.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.511+0000 7f6601409640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f65fc11be90 con 0x7f65fc1042a0 2024-09-11T08:01:39.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.511+0000 7f6601409640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f65fc11d090 con 0x7f65fc101530 2024-09-11T08:01:39.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.512+0000 7f65faffd640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7f65fc101530 0x7f65fc104670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:36020/0 (socket says 172.21.15.102:36020) 2024-09-11T08:01:39.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.512+0000 7f65faffd640 1 -- 172.21.15.102:0/154557077 learned_addr learned my addr 172.21.15.102:0/154557077 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:01:39.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.512+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1965168730 0 0) 0x7f65fc11d090 con 0x7f65fc101530 2024-09-11T08:01:39.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.512+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65e8003200 con 0x7f65fc101530 2024-09-11T08:01:39.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.512+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82320427 0 0) 0x7f65fc11ac90 con 0x7f65fc107e00 2024-09-11T08:01:39.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.512+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65fc11d090 con 0x7f65fc107e00 2024-09-11T08:01:39.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.512+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1572011692 0 0) 0x7f65e8003200 con 0x7f65fc101530 2024-09-11T08:01:39.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.512+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f65fc11ac90 con 0x7f65fc101530 2024-09-11T08:01:39.515 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.512+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f65ec0020e0 con 0x7f65fc101530 2024-09-11T08:01:39.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.512+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1033190706 0 0) 0x7f65fc11be90 con 0x7f65fc1042a0 2024-09-11T08:01:39.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.512+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65e8003200 con 0x7f65fc1042a0 2024-09-11T08:01:39.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.512+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2552936036 0 0) 0x7f65fc11d090 con 0x7f65fc107e00 2024-09-11T08:01:39.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.512+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f65fc11be90 con 0x7f65fc107e00 2024-09-11T08:01:39.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.512+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f65f4001bd0 con 0x7f65fc107e00 2024-09-11T08:01:39.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.512+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1419588131 0 0) 0x7f65fc11ac90 con 0x7f65fc101530 2024-09-11T08:01:39.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.512+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 >> v1:172.21.15.146:6789/0 conn(0x7f65fc1042a0 legacy=0x7f65fc104d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:39.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.512+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 >> v1:172.21.15.102:6789/0 conn(0x7f65fc107e00 legacy=0x7f65fc119450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:39.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.512+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65fc11e290 con 0x7f65fc101530 2024-09-11T08:01:39.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.512+0000 7f6601409640 1 -- 172.21.15.102:0/154557077 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65fc11ae80 con 0x7f65fc101530 2024-09-11T08:01:39.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.512+0000 7f6601409640 1 -- 172.21.15.102:0/154557077 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f65fc11b350 con 0x7f65fc101530 2024-09-11T08:01:39.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.513+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f65ec001af0 con 0x7f65fc101530 2024-09-11T08:01:39.516 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.513+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f65ec003630 con 0x7f65fc101530 2024-09-11T08:01:39.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.514+0000 7f6601409640 1 -- 172.21.15.102:0/154557077 --> v1:172.21.15.102:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f65dc000f20 con 0x7f65fc101530 2024-09-11T08:01:39.517 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.514+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7f65ec003850 con 0x7f65fc101530 2024-09-11T08:01:39.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.515+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 376332686 0 0) 0x7f65ec08a120 con 0x7f65fc101530 2024-09-11T08:01:39.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.515+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 --> v1:172.21.15.146:6812/2181809960 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f65e807e1f0 con 0x7f65e807a550 2024-09-11T08:01:39.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.515+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (unknown 1962166362 0 0) 0x7f65ec08a550 con 0x7f65fc101530 2024-09-11T08:01:39.518 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.516+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 <== osd.7 v1:172.21.15.146:6812/2181809960 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f65e807e1f0 con 0x7f65e807a550 2024-09-11T08:01:39.531 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:01:39.532 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:01:39.532 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:01:39.533 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.532+0000 7f6601409640 1 -- 172.21.15.102:0/154557077 --> v1:172.21.15.146:6812/2181809960 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f65dc002b10 con 0x7f65e807a550 2024-09-11T08:01:39.534 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.533+0000 7f65f37fe640 1 -- 172.21.15.102:0/154557077 <== osd.7 v1:172.21.15.146:6812/2181809960 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 2992730707) 0x7f65dc002b10 con 0x7f65e807a550 2024-09-11T08:01:39.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.534+0000 7f6601409640 1 -- 172.21.15.102:0/154557077 >> v1:172.21.15.146:6812/2181809960 conn(0x7f65e807a550 legacy=0x7f65e807c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:39.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.534+0000 7f6601409640 1 -- 172.21.15.102:0/154557077 >> v1:172.21.15.102:6800/4281557748 conn(0x7f65e806fcd0 legacy=0x7f65e8072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:39.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.534+0000 7f6601409640 1 -- 172.21.15.102:0/154557077 >> v1:172.21.15.102:6790/0 conn(0x7f65fc101530 legacy=0x7f65fc104670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:39.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.534+0000 7f65fb7fe640 1 -- 172.21.15.102:0/154557077 reap_dead start 2024-09-11T08:01:39.535 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.534+0000 7f6601409640 1 -- 172.21.15.102:0/154557077 shutdown_connections 2024-09-11T08:01:39.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.534+0000 7f6601409640 1 -- 172.21.15.102:0/154557077 >> 172.21.15.102:0/154557077 conn(0x7f65fc0752c0 msgr2=0x7f65fc10b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:39.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.534+0000 7f6601409640 1 -- 172.21.15.102:0/154557077 shutdown_connections 2024-09-11T08:01:39.536 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:39.534+0000 7f6601409640 1 -- 172.21.15.102:0/154557077 wait complete. 2024-09-11T08:01:39.684 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:01:39 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:01:39.987 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:39 smithi102 ceph-mon[29849]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:39.987 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:39 smithi102 ceph-mon[41371]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:40.011 INFO:teuthology.orchestra.run.smithi102.stdout:180388626451 2024-09-11T08:01:40.011 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-11T08:01:40.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:39 smithi146 ceph-mon[33158]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:40.510 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738414 got 34359738417 for osd.0 2024-09-11T08:01:40.510 DEBUG:teuthology.parallel:result is None 2024-09-11T08:01:40.745 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:40 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:01:40.745 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:40 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:01:41.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:40 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:01:41.883 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:41 smithi102 ceph-mon[41371]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:01:41.884 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:41 smithi102 ceph-mon[29849]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:01:42.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:41 smithi146 ceph-mon[33158]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:01:42.672 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:01:42 smithi102 podman[75955]: 2024-09-11 08:01:42.450880944 +0000 UTC m=+0.019035320 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-11T08:01:42.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.906+0000 7f7adc60e640 1 -- 172.21.15.102:0/4011525118 >> v1:172.21.15.102:6790/0 conn(0x7f7ad4107dd0 legacy=0x7f7ad410a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:42.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.906+0000 7f7adc60e640 1 -- 172.21.15.102:0/4011525118 shutdown_connections 2024-09-11T08:01:42.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.906+0000 7f7adc60e640 1 -- 172.21.15.102:0/4011525118 >> 172.21.15.102:0/4011525118 conn(0x7f7ad40f7000 msgr2=0x7f7ad40f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:42.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.906+0000 7f7adc60e640 1 -- 172.21.15.102:0/4011525118 shutdown_connections 2024-09-11T08:01:42.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.906+0000 7f7adc60e640 1 -- 172.21.15.102:0/4011525118 wait complete. 2024-09-11T08:01:42.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.906+0000 7f7adc60e640 1 Processor -- start 2024-09-11T08:01:42.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.906+0000 7f7adc60e640 1 -- start start 2024-09-11T08:01:42.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7adc60e640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7ad411bf90 con 0x7f7ad4107dd0 2024-09-11T08:01:42.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7adc60e640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7ad411d150 con 0x7f7ad4101500 2024-09-11T08:01:42.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7adc60e640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7ad411e350 con 0x7f7ad4104270 2024-09-11T08:01:42.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7ad9ba9640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7f7ad4104270 0x7f7ad4066430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:36034/0 (socket says 172.21.15.102:36034) 2024-09-11T08:01:42.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7ad9ba9640 1 -- 172.21.15.102:0/4155117987 learned_addr learned my addr 172.21.15.102:0/4155117987 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:01:42.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7adabab640 1 --1- 172.21.15.102:0/4155117987 >> v1:172.21.15.102:6789/0 conn(0x7f7ad4107dd0 0x7f7ad411a690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:41026/0 (socket says 172.21.15.102:41026) 2024-09-11T08:01:42.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3097552955 0 0) 0x7f7ad411bf90 con 0x7f7ad4107dd0 2024-09-11T08:01:42.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ac0003200 con 0x7f7ad4107dd0 2024-09-11T08:01:42.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4174394065 0 0) 0x7f7ad411e350 con 0x7f7ad4104270 2024-09-11T08:01:42.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ad411bf90 con 0x7f7ad4104270 2024-09-11T08:01:42.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1978123253 0 0) 0x7f7ad411d150 con 0x7f7ad4101500 2024-09-11T08:01:42.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ad411e350 con 0x7f7ad4101500 2024-09-11T08:01:42.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 448964219 0 0) 0x7f7ac0003200 con 0x7f7ad4107dd0 2024-09-11T08:01:42.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ad411d150 con 0x7f7ad4107dd0 2024-09-11T08:01:42.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f7acc002ed0 con 0x7f7ad4107dd0 2024-09-11T08:01:42.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3780100020 0 0) 0x7f7ad411bf90 con 0x7f7ad4104270 2024-09-11T08:01:42.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ac0003200 con 0x7f7ad4104270 2024-09-11T08:01:42.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f7ac4001910 con 0x7f7ad4104270 2024-09-11T08:01:42.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2944677309 0 0) 0x7f7ad411e350 con 0x7f7ad4101500 2024-09-11T08:01:42.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ad411bf90 con 0x7f7ad4101500 2024-09-11T08:01:42.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f7ad0001920 con 0x7f7ad4101500 2024-09-11T08:01:42.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3291898681 0 0) 0x7f7ad411d150 con 0x7f7ad4107dd0 2024-09-11T08:01:42.911 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 >> v1:172.21.15.102:6790/0 conn(0x7f7ad4104270 legacy=0x7f7ad4066430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:42.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 >> v1:172.21.15.146:6789/0 conn(0x7f7ad4101500 legacy=0x7f7ad4065d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:42.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad411f550 con 0x7f7ad4107dd0 2024-09-11T08:01:42.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.907+0000 7f7adc60e640 1 -- 172.21.15.102:0/4155117987 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad411e540 con 0x7f7ad4107dd0 2024-09-11T08:01:42.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.908+0000 7f7adc60e640 1 -- 172.21.15.102:0/4155117987 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad411e990 con 0x7f7ad4107dd0 2024-09-11T08:01:42.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.908+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f7acc002650 con 0x7f7ad4107dd0 2024-09-11T08:01:42.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.908+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f7acc003760 con 0x7f7ad4107dd0 2024-09-11T08:01:42.912 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.909+0000 7f7adc60e640 1 -- 172.21.15.102:0/4155117987 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab8005010 con 0x7f7ad4107dd0 2024-09-11T08:01:42.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.909+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7f7acc002200 con 0x7f7ad4107dd0 2024-09-11T08:01:42.913 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.910+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 376332686 0 0) 0x7f7acc08a1d0 con 0x7f7ad4107dd0 2024-09-11T08:01:42.918 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:42.917+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f7acc056f20 con 0x7f7ad4107dd0 2024-09-11T08:01:43.014 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:01:42 smithi102 podman[75955]: 2024-09-11 08:01:42.739954833 +0000 UTC m=+0.308109201 volume create faf3a9f569c358e0c7e27efb37dbeb58e1bfa39359c714adfe3ca8ca529c9ee7 2024-09-11T08:01:43.080 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:43.079+0000 7f7adc60e640 1 -- 172.21.15.102:0/4155117987 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7ab80052c0 con 0x7f7ad4107dd0 2024-09-11T08:01:43.081 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:43.079+0000 7f7acaffd640 1 -- 172.21.15.102:0/4155117987 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (unknown 92182286 0 3693870858) 0x7f7acc0568c0 con 0x7f7ad4107dd0 2024-09-11T08:01:43.081 INFO:teuthology.orchestra.run.smithi102.stdout:77309411369 2024-09-11T08:01:43.084 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:43.081+0000 7f7adc60e640 1 -- 172.21.15.102:0/4155117987 >> v1:172.21.15.102:6800/4281557748 conn(0x7f7ac006fd80 legacy=0x7f7ac0072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:43.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:43.081+0000 7f7adc60e640 1 -- 172.21.15.102:0/4155117987 >> v1:172.21.15.102:6789/0 conn(0x7f7ad4107dd0 legacy=0x7f7ad411a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:43.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:43.081+0000 7f7adc60e640 1 -- 172.21.15.102:0/4155117987 shutdown_connections 2024-09-11T08:01:43.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:43.081+0000 7f7adc60e640 1 -- 172.21.15.102:0/4155117987 >> 172.21.15.102:0/4155117987 conn(0x7f7ad40f7000 msgr2=0x7f7ad40fbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:43.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:43.081+0000 7f7adc60e640 1 -- 172.21.15.102:0/4155117987 shutdown_connections 2024-09-11T08:01:43.085 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:43.081+0000 7f7adc60e640 1 -- 172.21.15.102:0/4155117987 wait complete. 2024-09-11T08:01:43.395 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:01:43 smithi102 podman[75955]: 2024-09-11 08:01:43.028283368 +0000 UTC m=+0.596437739 container create afe38f922c4b48b21bfe4450e84cf647c6d653b066ab106dde766522c623c0a6 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:01:43.840 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:43 smithi102 ceph-mon[41371]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:43.841 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:43 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4155117987' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T08:01:43.841 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:43 smithi102 ceph-mon[29849]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:43.841 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:43 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4155117987' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T08:01:43.865 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411366 got 77309411369 for osd.2 2024-09-11T08:01:43.866 DEBUG:teuthology.parallel:result is None 2024-09-11T08:01:44.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:43 smithi146 ceph-mon[33158]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:44.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:43 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4155117987' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T08:01:44.749 INFO:teuthology.orchestra.run.smithi102.stdout:206158430224 2024-09-11T08:01:44.749 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-11T08:01:44.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:44 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:01:44.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:44 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:01:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:01:44.981 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:44 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:01:44.982 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:44 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:01:45.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.166+0000 7f7a484bc640 1 -- 172.21.15.102:0/4254553523 >> v1:172.21.15.102:6789/0 conn(0x7f7a40107dd0 legacy=0x7f7a4010a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:45.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.166+0000 7f7a484bc640 1 -- 172.21.15.102:0/4254553523 shutdown_connections 2024-09-11T08:01:45.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.166+0000 7f7a484bc640 1 -- 172.21.15.102:0/4254553523 >> 172.21.15.102:0/4254553523 conn(0x7f7a400f7060 msgr2=0x7f7a400f9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:45.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.167+0000 7f7a484bc640 1 -- 172.21.15.102:0/4254553523 shutdown_connections 2024-09-11T08:01:45.190 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.167+0000 7f7a484bc640 1 -- 172.21.15.102:0/4254553523 wait complete. 2024-09-11T08:01:45.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.167+0000 7f7a484bc640 1 Processor -- start 2024-09-11T08:01:45.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.167+0000 7f7a484bc640 1 -- start start 2024-09-11T08:01:45.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.167+0000 7f7a484bc640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7a40113590 con 0x7f7a40101500 2024-09-11T08:01:45.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.167+0000 7f7a484bc640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7a40107740 con 0x7f7a40107dd0 2024-09-11T08:01:45.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.167+0000 7f7a484bc640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7a401078d0 con 0x7f7a40104270 2024-09-11T08:01:45.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.167+0000 7f7a46258640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f7a40101500 0x7f7a40107030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:41052/0 (socket says 172.21.15.102:41052) 2024-09-11T08:01:45.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.167+0000 7f7a46258640 1 -- 172.21.15.102:0/2691208169 learned_addr learned my addr 172.21.15.102:0/2691208169 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:01:45.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.168+0000 7f7a3effd640 1 -- 172.21.15.102:0/2691208169 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4145423628 0 0) 0x7f7a40113590 con 0x7f7a40101500 2024-09-11T08:01:45.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.168+0000 7f7a3effd640 1 -- 172.21.15.102:0/2691208169 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a28003200 con 0x7f7a40101500 2024-09-11T08:01:45.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.168+0000 7f7a3effd640 1 -- 172.21.15.102:0/2691208169 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3237832661 0 0) 0x7f7a401078d0 con 0x7f7a40104270 2024-09-11T08:01:45.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.168+0000 7f7a3effd640 1 -- 172.21.15.102:0/2691208169 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a40113590 con 0x7f7a40104270 2024-09-11T08:01:45.191 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.168+0000 7f7a3effd640 1 -- 172.21.15.102:0/2691208169 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3096061373 0 0) 0x7f7a40107740 con 0x7f7a40107dd0 2024-09-11T08:01:45.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.168+0000 7f7a3effd640 1 -- 172.21.15.102:0/2691208169 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a401078d0 con 0x7f7a40107dd0 2024-09-11T08:01:45.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.168+0000 7f7a3effd640 1 -- 172.21.15.102:0/2691208169 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3160904911 0 0) 0x7f7a40113590 con 0x7f7a40104270 2024-09-11T08:01:45.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.168+0000 7f7a3effd640 1 -- 172.21.15.102:0/2691208169 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7a40107740 con 0x7f7a40104270 2024-09-11T08:01:45.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.168+0000 7f7a3effd640 1 -- 172.21.15.102:0/2691208169 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f7a340016a0 con 0x7f7a40104270 2024-09-11T08:01:45.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.168+0000 7f7a3effd640 1 -- 172.21.15.102:0/2691208169 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 790638959 0 0) 0x7f7a28003200 con 0x7f7a40101500 2024-09-11T08:01:45.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.168+0000 7f7a3effd640 1 -- 172.21.15.102:0/2691208169 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7a40113590 con 0x7f7a40101500 2024-09-11T08:01:45.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.168+0000 7f7a3effd640 1 -- 172.21.15.102:0/2691208169 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f7a380018c0 con 0x7f7a40101500 2024-09-11T08:01:45.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.168+0000 7f7a3effd640 1 -- 172.21.15.102:0/2691208169 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3792901715 0 0) 0x7f7a40113590 con 0x7f7a40101500 2024-09-11T08:01:45.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.168+0000 7f7a3effd640 1 -- 172.21.15.102:0/2691208169 >> v1:172.21.15.102:6790/0 conn(0x7f7a40104270 legacy=0x7f7a4011dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:45.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.168+0000 7f7a3effd640 1 -- 172.21.15.102:0/2691208169 >> v1:172.21.15.146:6789/0 conn(0x7f7a40107dd0 legacy=0x7f7a40121710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:45.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.168+0000 7f7a3effd640 1 -- 172.21.15.102:0/2691208169 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a40107aa0 con 0x7f7a40101500 2024-09-11T08:01:45.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.168+0000 7f7a484bc640 1 -- 172.21.15.102:0/2691208169 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a40122e30 con 0x7f7a40101500 2024-09-11T08:01:45.192 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.168+0000 7f7a484bc640 1 -- 172.21.15.102:0/2691208169 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a401232a0 con 0x7f7a40101500 2024-09-11T08:01:45.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.168+0000 7f7a3effd640 1 -- 172.21.15.102:0/2691208169 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f7a38001f30 con 0x7f7a40101500 2024-09-11T08:01:45.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.168+0000 7f7a3effd640 1 -- 172.21.15.102:0/2691208169 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f7a38005a10 con 0x7f7a40101500 2024-09-11T08:01:45.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.169+0000 7f7a484bc640 1 -- 172.21.15.102:0/2691208169 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a2c005010 con 0x7f7a40101500 2024-09-11T08:01:45.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.170+0000 7f7a3effd640 1 -- 172.21.15.102:0/2691208169 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7f7a38001ae0 con 0x7f7a40101500 2024-09-11T08:01:45.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.170+0000 7f7a3effd640 1 -- 172.21.15.102:0/2691208169 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 376332686 0 0) 0x7f7a38089e10 con 0x7f7a40101500 2024-09-11T08:01:45.193 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.174+0000 7f7a3effd640 1 -- 172.21.15.102:0/2691208169 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f7a38056b60 con 0x7f7a40101500 2024-09-11T08:01:45.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.336+0000 7f7a484bc640 1 -- 172.21.15.102:0/2691208169 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f7a2c0052c0 con 0x7f7a40101500 2024-09-11T08:01:45.338 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.336+0000 7f7a3effd640 1 -- 172.21.15.102:0/2691208169 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (unknown 1823589948 0 1469866018) 0x7f7a38056500 con 0x7f7a40101500 2024-09-11T08:01:45.339 INFO:teuthology.orchestra.run.smithi102.stdout:128849018912 2024-09-11T08:01:45.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.339+0000 7f7a484bc640 1 -- 172.21.15.102:0/2691208169 >> v1:172.21.15.102:6800/4281557748 conn(0x7f7a2806fac0 legacy=0x7f7a28071f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:45.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.339+0000 7f7a484bc640 1 -- 172.21.15.102:0/2691208169 >> v1:172.21.15.102:6789/0 conn(0x7f7a40101500 legacy=0x7f7a40107030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:45.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.339+0000 7f7a484bc640 1 -- 172.21.15.102:0/2691208169 shutdown_connections 2024-09-11T08:01:45.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.339+0000 7f7a484bc640 1 -- 172.21.15.102:0/2691208169 >> 172.21.15.102:0/2691208169 conn(0x7f7a400f7060 msgr2=0x7f7a40066430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:45.341 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.339+0000 7f7a484bc640 1 -- 172.21.15.102:0/2691208169 shutdown_connections 2024-09-11T08:01:45.342 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:45.339+0000 7f7a484bc640 1 -- 172.21.15.102:0/2691208169 wait complete. 2024-09-11T08:01:45.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:45 smithi102 ceph-mon[29849]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:45.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:45 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2691208169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T08:01:45.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:45 smithi102 ceph-mon[41371]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:45.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:45 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2691208169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T08:01:46.180 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018908 got 128849018912 for osd.4 2024-09-11T08:01:46.180 DEBUG:teuthology.parallel:result is None 2024-09-11T08:01:46.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:45 smithi146 ceph-mon[33158]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:46.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:45 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2691208169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T08:01:46.937 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:01:46 smithi102 podman[75955]: 2024-09-11 08:01:46.645403204 +0000 UTC m=+4.213557581 container init afe38f922c4b48b21bfe4450e84cf647c6d653b066ab106dde766522c623c0a6 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:01:46.937 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:01:46 smithi102 podman[75955]: 2024-09-11 08:01:46.653906702 +0000 UTC m=+4.222061073 container start afe38f922c4b48b21bfe4450e84cf647c6d653b066ab106dde766522c623c0a6 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:01:46.937 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:01:46 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[76080]: ts=2024-09-11T08:01:46.667Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-11T08:01:46.937 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:01:46 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[76080]: ts=2024-09-11T08:01:46.667Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-11T08:01:46.937 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:01:46 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[76080]: ts=2024-09-11T08:01:46.668Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.102 port=9094 2024-09-11T08:01:46.937 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:01:46 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[76080]: ts=2024-09-11T08:01:46.669Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-11T08:01:46.937 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:01:46 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[76080]: ts=2024-09-11T08:01:46.693Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T08:01:46.937 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:01:46 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[76080]: ts=2024-09-11T08:01:46.693Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T08:01:46.937 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:01:46 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[76080]: ts=2024-09-11T08:01:46.696Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-11T08:01:46.937 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:01:46 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[76080]: ts=2024-09-11T08:01:46.696Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-11T08:01:46.938 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:01:46 smithi102 bash[75955]: afe38f922c4b48b21bfe4450e84cf647c6d653b066ab106dde766522c623c0a6 2024-09-11T08:01:46.938 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:01:46 smithi102 systemd[1]: Started Ceph alertmanager.a for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T08:01:47.236 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:01:47] ENGINE Bus STOPPING 2024-09-11T08:01:47.237 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:01:47] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:01:47.237 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:01:47] ENGINE Bus STOPPED 2024-09-11T08:01:47.237 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:01:47] ENGINE Bus STARTING 2024-09-11T08:01:47.237 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:01:47] ENGINE Serving on http://:::9283 2024-09-11T08:01:47.237 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:01:47] ENGINE Bus STARTED 2024-09-11T08:01:47.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.234+0000 7f04fd408640 1 -- 172.21.15.102:0/2716008020 >> v1:172.21.15.102:6790/0 conn(0x7f04f8108ee0 legacy=0x7f04f810b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:47.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.234+0000 7f04fd408640 1 -- 172.21.15.102:0/2716008020 shutdown_connections 2024-09-11T08:01:47.274 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.234+0000 7f04fd408640 1 -- 172.21.15.102:0/2716008020 >> 172.21.15.102:0/2716008020 conn(0x7f04f80f8150 msgr2=0x7f04f80fa560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:47.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.234+0000 7f04fd408640 1 -- 172.21.15.102:0/2716008020 shutdown_connections 2024-09-11T08:01:47.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.234+0000 7f04fd408640 1 -- 172.21.15.102:0/2716008020 wait complete. 2024-09-11T08:01:47.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.235+0000 7f04fd408640 1 Processor -- start 2024-09-11T08:01:47.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.235+0000 7f04fd408640 1 -- start start 2024-09-11T08:01:47.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.235+0000 7f04fd408640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f04f8101b90 con 0x7f04f8102590 2024-09-11T08:01:47.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.235+0000 7f04fd408640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f04f8101d80 con 0x7f04f8105350 2024-09-11T08:01:47.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.235+0000 7f04fd408640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f04f8101f70 con 0x7f04f8108ee0 2024-09-11T08:01:47.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.235+0000 7f04f6ffd640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f04f8102590 0x7f04f8101360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:41070/0 (socket says 172.21.15.102:41070) 2024-09-11T08:01:47.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.235+0000 7f04f6ffd640 1 -- 172.21.15.102:0/3227157506 learned_addr learned my addr 172.21.15.102:0/3227157506 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:01:47.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.235+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1220613942 0 0) 0x7f04f8101f70 con 0x7f04f8108ee0 2024-09-11T08:01:47.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.235+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04e00034e0 con 0x7f04f8108ee0 2024-09-11T08:01:47.275 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.235+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3606977007 0 0) 0x7f04f8101b90 con 0x7f04f8102590 2024-09-11T08:01:47.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.235+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04f8101f70 con 0x7f04f8102590 2024-09-11T08:01:47.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.235+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1025877535 0 0) 0x7f04f8101d80 con 0x7f04f8105350 2024-09-11T08:01:47.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.235+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04f8101b90 con 0x7f04f8105350 2024-09-11T08:01:47.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.236+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4191049087 0 0) 0x7f04e00034e0 con 0x7f04f8108ee0 2024-09-11T08:01:47.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.236+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f04f8101d80 con 0x7f04f8108ee0 2024-09-11T08:01:47.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.236+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f04f0001510 con 0x7f04f8108ee0 2024-09-11T08:01:47.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.236+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 447238311 0 0) 0x7f04f8101b90 con 0x7f04f8105350 2024-09-11T08:01:47.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.236+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f04e00034e0 con 0x7f04f8105350 2024-09-11T08:01:47.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.236+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2805740505 0 0) 0x7f04f8101f70 con 0x7f04f8102590 2024-09-11T08:01:47.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.236+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f04f8101b90 con 0x7f04f8102590 2024-09-11T08:01:47.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.236+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f04ec001830 con 0x7f04f8105350 2024-09-11T08:01:47.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.236+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f04e8001c80 con 0x7f04f8102590 2024-09-11T08:01:47.276 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.236+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3486311180 0 0) 0x7f04f8101d80 con 0x7f04f8108ee0 2024-09-11T08:01:47.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.236+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 >> v1:172.21.15.146:6789/0 conn(0x7f04f8105350 legacy=0x7f04f8119900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:47.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.236+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 >> v1:172.21.15.102:6789/0 conn(0x7f04f8102590 legacy=0x7f04f8101360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:47.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.236+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f81217c0 con 0x7f04f8108ee0 2024-09-11T08:01:47.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.236+0000 7f04fd408640 1 -- 172.21.15.102:0/3227157506 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f811e7b0 con 0x7f04f8108ee0 2024-09-11T08:01:47.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.236+0000 7f04fd408640 1 -- 172.21.15.102:0/3227157506 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f811ec00 con 0x7f04f8108ee0 2024-09-11T08:01:47.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.236+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f04f0001ab0 con 0x7f04f8108ee0 2024-09-11T08:01:47.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.237+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f04f00034a0 con 0x7f04f8108ee0 2024-09-11T08:01:47.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.237+0000 7f04fd408640 1 -- 172.21.15.102:0/3227157506 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e4005010 con 0x7f04f8108ee0 2024-09-11T08:01:47.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.238+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7f04f00198c0 con 0x7f04f8108ee0 2024-09-11T08:01:47.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.238+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 376332686 0 0) 0x7f04f0087710 con 0x7f04f8108ee0 2024-09-11T08:01:47.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.244+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f04f0054da0 con 0x7f04f8108ee0 2024-09-11T08:01:47.281 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:01:47.282 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:01:47.325 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:01:47.330 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:01:47.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.426+0000 7f04fd408640 1 -- 172.21.15.102:0/3227157506 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f04e40052c0 con 0x7f04f8108ee0 2024-09-11T08:01:47.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.427+0000 7f04df7fe640 1 -- 172.21.15.102:0/3227157506 <== mon.2 v1:172.21.15.102:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (unknown 832126871 0 1791578311) 0x7f04f0054bc0 con 0x7f04f8108ee0 2024-09-11T08:01:47.429 INFO:teuthology.orchestra.run.smithi102.stdout:55834574895 2024-09-11T08:01:47.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.429+0000 7f04fd408640 1 -- 172.21.15.102:0/3227157506 >> v1:172.21.15.102:6800/4281557748 conn(0x7f04e006fd70 legacy=0x7f04e0072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:47.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.429+0000 7f04fd408640 1 -- 172.21.15.102:0/3227157506 >> v1:172.21.15.102:6790/0 conn(0x7f04f8108ee0 legacy=0x7f04f811d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:47.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.429+0000 7f04fd408640 1 -- 172.21.15.102:0/3227157506 shutdown_connections 2024-09-11T08:01:47.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.429+0000 7f04fd408640 1 -- 172.21.15.102:0/3227157506 >> 172.21.15.102:0/3227157506 conn(0x7f04f80f8150 msgr2=0x7f04f810bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:47.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.429+0000 7f04fd408640 1 -- 172.21.15.102:0/3227157506 shutdown_connections 2024-09-11T08:01:47.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:47.429+0000 7f04fd408640 1 -- 172.21.15.102:0/3227157506 wait complete. 2024-09-11T08:01:47.819 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-mon[41371]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:01:47.819 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:47.819 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:47.819 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:47.819 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:47.819 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:47.819 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:47.819 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:01:47.819 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:47.820 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3227157506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T08:01:47.820 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-mon[29849]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:01:47.820 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:47.820 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:47.820 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:47.820 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:47.820 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:47.820 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:47.820 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:01:47.820 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:47.820 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3227157506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T08:01:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:47 smithi146 ceph-mon[33158]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:01:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:47 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:47 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:47 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:47 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:47 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:47 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:47 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:01:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:47 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3227157506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T08:01:48.834 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:48 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:01:48.834 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:48 smithi102 ceph-mon[29849]: Deploying daemon grafana.a on smithi146 2024-09-11T08:01:48.834 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:48 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:01:48.834 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:48 smithi102 ceph-mon[41371]: Deploying daemon grafana.a on smithi146 2024-09-11T08:01:48.835 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:01:48 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[76080]: ts=2024-09-11T08:01:48.670Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000638675s 2024-09-11T08:01:49.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:48 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T08:01:49.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:48 smithi146 ceph-mon[33158]: Deploying daemon grafana.a on smithi146 2024-09-11T08:01:49.258 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574890 got 55834574895 for osd.1 2024-09-11T08:01:49.258 DEBUG:teuthology.parallel:result is None 2024-09-11T08:01:49.684 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:01:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:01:49.846 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:49 smithi102 ceph-mon[41371]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:49.846 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:49 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:49.846 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:49 smithi102 ceph-mon[29849]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:49.846 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:49 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:49 smithi146 ceph-mon[33158]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:49 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:50.988 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:50 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:01:50.988 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:50 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:01:51.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:50 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:01:52.063 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:51 smithi102 ceph-mon[41371]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:01:52.063 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:51 smithi102 ceph-mon[29849]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:01:52.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:51 smithi146 ceph-mon[33158]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:01:52.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.256+0000 7f055907a640 1 -- 172.21.15.102:0/2361346465 >> v1:172.21.15.102:6789/0 conn(0x7f0554105570 legacy=0x7f0554107950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:52.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.256+0000 7f055907a640 1 -- 172.21.15.102:0/2361346465 shutdown_connections 2024-09-11T08:01:52.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.256+0000 7f055907a640 1 -- 172.21.15.102:0/2361346465 >> 172.21.15.102:0/2361346465 conn(0x7f0554074cb0 msgr2=0x7f05540f8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:52.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.256+0000 7f055907a640 1 -- 172.21.15.102:0/2361346465 shutdown_connections 2024-09-11T08:01:52.289 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.256+0000 7f055907a640 1 -- 172.21.15.102:0/2361346465 wait complete. 2024-09-11T08:01:52.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.256+0000 7f055907a640 1 Processor -- start 2024-09-11T08:01:52.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.257+0000 7f055907a640 1 -- start start 2024-09-11T08:01:52.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.257+0000 7f055907a640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f055411c170 con 0x7f0554102840 2024-09-11T08:01:52.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.257+0000 7f055907a640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f055411d330 con 0x7f0554105570 2024-09-11T08:01:52.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.257+0000 7f055907a640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f055411e530 con 0x7f0554109130 2024-09-11T08:01:52.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.257+0000 7f0552d9d640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f0554102840 0x7f05540fe850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:43418/0 (socket says 172.21.15.102:43418) 2024-09-11T08:01:52.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.257+0000 7f0552d9d640 1 -- 172.21.15.102:0/453528146 learned_addr learned my addr 172.21.15.102:0/453528146 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:01:52.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.257+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510775383 0 0) 0x7f055411e530 con 0x7f0554109130 2024-09-11T08:01:52.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.257+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05340034e0 con 0x7f0554109130 2024-09-11T08:01:52.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.257+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1185122423 0 0) 0x7f055411c170 con 0x7f0554102840 2024-09-11T08:01:52.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.257+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f055411e530 con 0x7f0554102840 2024-09-11T08:01:52.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.257+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2425730831 0 0) 0x7f055411d330 con 0x7f0554105570 2024-09-11T08:01:52.290 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.257+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f055411c170 con 0x7f0554105570 2024-09-11T08:01:52.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.257+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1472590800 0 0) 0x7f05340034e0 con 0x7f0554109130 2024-09-11T08:01:52.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.257+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f055411d330 con 0x7f0554109130 2024-09-11T08:01:52.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.257+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f054c001d50 con 0x7f0554109130 2024-09-11T08:01:52.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.257+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1309678648 0 0) 0x7f055411e530 con 0x7f0554102840 2024-09-11T08:01:52.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.257+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f05340034e0 con 0x7f0554102840 2024-09-11T08:01:52.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.257+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f0544001900 con 0x7f0554102840 2024-09-11T08:01:52.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.258+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1734335202 0 0) 0x7f055411c170 con 0x7f0554105570 2024-09-11T08:01:52.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.258+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f055411e530 con 0x7f0554105570 2024-09-11T08:01:52.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.258+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f053c001510 con 0x7f0554105570 2024-09-11T08:01:52.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.258+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2158232435 0 0) 0x7f055411d330 con 0x7f0554109130 2024-09-11T08:01:52.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.258+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 >> v1:172.21.15.146:6789/0 conn(0x7f0554105570 legacy=0x7f05540fef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:52.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.258+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 >> v1:172.21.15.102:6789/0 conn(0x7f0554102840 legacy=0x7f05540fe850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:52.291 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.258+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055411f730 con 0x7f0554109130 2024-09-11T08:01:52.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.258+0000 7f055907a640 1 -- 172.21.15.102:0/453528146 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f055411c360 con 0x7f0554109130 2024-09-11T08:01:52.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.258+0000 7f055907a640 1 -- 172.21.15.102:0/453528146 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f055411c7b0 con 0x7f0554109130 2024-09-11T08:01:52.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.258+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f054c002670 con 0x7f0554109130 2024-09-11T08:01:52.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.258+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f054c003580 con 0x7f0554109130 2024-09-11T08:01:52.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.259+0000 7f055907a640 1 -- 172.21.15.102:0/453528146 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0538005010 con 0x7f0554109130 2024-09-11T08:01:52.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.259+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7f054c002220 con 0x7f0554109130 2024-09-11T08:01:52.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.260+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 376332686 0 0) 0x7f054c0877f0 con 0x7f0554109130 2024-09-11T08:01:52.292 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.264+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f054c054540 con 0x7f0554109130 2024-09-11T08:01:52.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.426+0000 7f055907a640 1 -- 172.21.15.102:0/453528146 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f05380052c0 con 0x7f0554109130 2024-09-11T08:01:52.429 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.427+0000 7f054bfff640 1 -- 172.21.15.102:0/453528146 <== mon.2 v1:172.21.15.102:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (unknown 1274345170 0 2258996860) 0x7f054c053ee0 con 0x7f0554109130 2024-09-11T08:01:52.429 INFO:teuthology.orchestra.run.smithi102.stdout:180388626455 2024-09-11T08:01:52.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.429+0000 7f055907a640 1 -- 172.21.15.102:0/453528146 >> v1:172.21.15.102:6800/4281557748 conn(0x7f053406fe80 legacy=0x7f0534072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:52.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.429+0000 7f055907a640 1 -- 172.21.15.102:0/453528146 >> v1:172.21.15.102:6790/0 conn(0x7f0554109130 legacy=0x7f055411a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:52.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.429+0000 7f055907a640 1 -- 172.21.15.102:0/453528146 shutdown_connections 2024-09-11T08:01:52.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.429+0000 7f055907a640 1 -- 172.21.15.102:0/453528146 >> 172.21.15.102:0/453528146 conn(0x7f0554074cb0 msgr2=0x7f055410beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:52.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.429+0000 7f055907a640 1 -- 172.21.15.102:0/453528146 shutdown_connections 2024-09-11T08:01:52.431 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.429+0000 7f055907a640 1 -- 172.21.15.102:0/453528146 wait complete. 2024-09-11T08:01:52.934 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:52 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/453528146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T08:01:52.935 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:52 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/453528146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T08:01:53.143 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626451 got 180388626455 for osd.6 2024-09-11T08:01:53.144 DEBUG:teuthology.parallel:result is None 2024-09-11T08:01:53.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:52 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/453528146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T08:01:53.797 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:53 smithi102 ceph-mon[41371]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:53.797 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:53 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3453576171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T08:01:53.797 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:53 smithi102 ceph-mon[29849]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:53.797 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:53 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3453576171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T08:01:53.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.932+0000 7f78276e7640 1 -- 172.21.15.102:0/3823682988 >> v1:172.21.15.102:6789/0 conn(0x7f7820102590 legacy=0x7f7820102960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:53.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.932+0000 7f78276e7640 1 -- 172.21.15.102:0/3823682988 shutdown_connections 2024-09-11T08:01:53.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.932+0000 7f78276e7640 1 -- 172.21.15.102:0/3823682988 >> 172.21.15.102:0/3823682988 conn(0x7f78200f81b0 msgr2=0x7f78200fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:53.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.932+0000 7f78276e7640 1 -- 172.21.15.102:0/3823682988 shutdown_connections 2024-09-11T08:01:53.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.932+0000 7f78276e7640 1 -- 172.21.15.102:0/3823682988 wait complete. 2024-09-11T08:01:53.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.933+0000 7f78276e7640 1 Processor -- start 2024-09-11T08:01:53.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.933+0000 7f78276e7640 1 -- start start 2024-09-11T08:01:53.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.933+0000 7f78276e7640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f782011bfd0 con 0x7f7820108ee0 2024-09-11T08:01:53.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.933+0000 7f78276e7640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f782011d1d0 con 0x7f7820102590 2024-09-11T08:01:53.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.933+0000 7f78276e7640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f782011e3d0 con 0x7f7820105350 2024-09-11T08:01:53.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.933+0000 7f7825483640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7f7820102590 0x7f78200fe760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.102:43622/0 (socket says 172.21.15.102:43622) 2024-09-11T08:01:53.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.933+0000 7f7825483640 1 -- 172.21.15.102:0/3453576171 learned_addr learned my addr 172.21.15.102:0/3453576171 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:01:53.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.934+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1087833643 0 0) 0x7f782011d1d0 con 0x7f7820102590 2024-09-11T08:01:53.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.934+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f780c0034e0 con 0x7f7820102590 2024-09-11T08:01:53.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.934+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 308574656 0 0) 0x7f782011e3d0 con 0x7f7820105350 2024-09-11T08:01:53.931 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.934+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f782011d1d0 con 0x7f7820105350 2024-09-11T08:01:53.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.934+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2730477854 0 0) 0x7f782011bfd0 con 0x7f7820108ee0 2024-09-11T08:01:53.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.934+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f782011e3d0 con 0x7f7820108ee0 2024-09-11T08:01:53.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.934+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 345609868 0 0) 0x7f780c0034e0 con 0x7f7820102590 2024-09-11T08:01:53.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.934+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f782011bfd0 con 0x7f7820102590 2024-09-11T08:01:53.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.934+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f781c002d50 con 0x7f7820102590 2024-09-11T08:01:53.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.934+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3016776798 0 0) 0x7f782011e3d0 con 0x7f7820108ee0 2024-09-11T08:01:53.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.934+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f780c0034e0 con 0x7f7820108ee0 2024-09-11T08:01:53.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.934+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f7818001b40 con 0x7f7820108ee0 2024-09-11T08:01:53.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.934+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 555585455 0 0) 0x7f782011d1d0 con 0x7f7820105350 2024-09-11T08:01:53.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.934+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f782011e3d0 con 0x7f7820105350 2024-09-11T08:01:53.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.934+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f78140018c0 con 0x7f7820105350 2024-09-11T08:01:53.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.934+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 <== mon.1 v1:172.21.15.146:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 109054941 0 0) 0x7f782011bfd0 con 0x7f7820102590 2024-09-11T08:01:53.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.934+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 >> v1:172.21.15.102:6790/0 conn(0x7f7820105350 legacy=0x7f78200fee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:53.932 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.934+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 >> v1:172.21.15.102:6789/0 conn(0x7f7820108ee0 legacy=0x7f782011a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:53.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.934+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782011f5d0 con 0x7f7820102590 2024-09-11T08:01:53.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.934+0000 7f78276e7640 1 -- 172.21.15.102:0/3453576171 --> v1:172.21.15.146:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f782011e5c0 con 0x7f7820102590 2024-09-11T08:01:53.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.934+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 <== mon.1 v1:172.21.15.146:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f781c0024a0 con 0x7f7820102590 2024-09-11T08:01:53.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.934+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 <== mon.1 v1:172.21.15.146:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f781c0036b0 con 0x7f7820102590 2024-09-11T08:01:53.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.934+0000 7f78276e7640 1 -- 172.21.15.102:0/3453576171 --> v1:172.21.15.146:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f782011ea90 con 0x7f7820102590 2024-09-11T08:01:53.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.936+0000 7f78276e7640 1 -- 172.21.15.102:0/3453576171 --> v1:172.21.15.146:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78200fdd10 con 0x7f7820102590 2024-09-11T08:01:53.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.936+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 <== mon.1 v1:172.21.15.146:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7f781c002050 con 0x7f7820102590 2024-09-11T08:01:53.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.937+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 <== mon.1 v1:172.21.15.146:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 376332686 0 0) 0x7f781c08a1e0 con 0x7f7820102590 2024-09-11T08:01:53.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:52.941+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 <== mon.1 v1:172.21.15.146:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f781c056ed0 con 0x7f7820102590 2024-09-11T08:01:53.933 INFO:teuthology.orchestra.run.smithi102.stdout:206158430227 2024-09-11T08:01:53.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:53.104+0000 7f78276e7640 1 -- 172.21.15.102:0/3453576171 --> v1:172.21.15.146:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f78200ff8b0 con 0x7f7820102590 2024-09-11T08:01:53.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:53.105+0000 7f7811ffb640 1 -- 172.21.15.102:0/3453576171 <== mon.1 v1:172.21.15.146:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (unknown 1482059429 0 499304692) 0x7f781c056870 con 0x7f7820102590 2024-09-11T08:01:53.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:53.106+0000 7f78276e7640 1 -- 172.21.15.102:0/3453576171 >> v1:172.21.15.102:6800/4281557748 conn(0x7f780c06ff30 legacy=0x7f780c0723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:53.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:53.106+0000 7f78276e7640 1 -- 172.21.15.102:0/3453576171 >> v1:172.21.15.146:6789/0 conn(0x7f7820102590 legacy=0x7f78200fe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:53.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:53.106+0000 7f78276e7640 1 -- 172.21.15.102:0/3453576171 shutdown_connections 2024-09-11T08:01:53.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:53.106+0000 7f78276e7640 1 -- 172.21.15.102:0/3453576171 >> 172.21.15.102:0/3453576171 conn(0x7f78200f81b0 msgr2=0x7f7820109360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:53.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:53.107+0000 7f78276e7640 1 -- 172.21.15.102:0/3453576171 shutdown_connections 2024-09-11T08:01:53.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:53.107+0000 7f78276e7640 1 -- 172.21.15.102:0/3453576171 wait complete. 2024-09-11T08:01:54.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:53 smithi146 ceph-mon[33158]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:54.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:53 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3453576171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T08:01:54.459 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430224 got 206158430227 for osd.7 2024-09-11T08:01:54.459 DEBUG:teuthology.parallel:result is None 2024-09-11T08:01:54.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.641+0000 7fe298926640 1 -- 172.21.15.102:0/3001186459 >> v1:172.21.15.102:6789/0 conn(0x7fe294104210 legacy=0x7fe2941065f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:54.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.641+0000 7fe298926640 1 -- 172.21.15.102:0/3001186459 shutdown_connections 2024-09-11T08:01:54.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.641+0000 7fe298926640 1 -- 172.21.15.102:0/3001186459 >> 172.21.15.102:0/3001186459 conn(0x7fe294074a50 msgr2=0x7fe2940f8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:54.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.641+0000 7fe298926640 1 -- 172.21.15.102:0/3001186459 shutdown_connections 2024-09-11T08:01:54.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.641+0000 7fe298926640 1 -- 172.21.15.102:0/3001186459 wait complete. 2024-09-11T08:01:54.651 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.641+0000 7fe298926640 1 Processor -- start 2024-09-11T08:01:54.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.642+0000 7fe298926640 1 -- start start 2024-09-11T08:01:54.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.642+0000 7fe298926640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe29411aea0 con 0x7fe294107e30 2024-09-11T08:01:54.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.642+0000 7fe298926640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe29411c0a0 con 0x7fe294104210 2024-09-11T08:01:54.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.642+0000 7fe298926640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe29411d2a0 con 0x7fe2941004b0 2024-09-11T08:01:54.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.642+0000 7fe29259c640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7fe2941004b0 0x7fe294104780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:35266/0 (socket says 172.21.15.102:35266) 2024-09-11T08:01:54.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.642+0000 7fe29259c640 1 -- 172.21.15.102:0/639186890 learned_addr learned my addr 172.21.15.102:0/639186890 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:01:54.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.642+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 377291145 0 0) 0x7fe29411d2a0 con 0x7fe2941004b0 2024-09-11T08:01:54.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.642+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2800034e0 con 0x7fe2941004b0 2024-09-11T08:01:54.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.642+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 759246218 0 0) 0x7fe29411aea0 con 0x7fe294107e30 2024-09-11T08:01:54.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.642+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe29411d2a0 con 0x7fe294107e30 2024-09-11T08:01:54.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.642+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4201890546 0 0) 0x7fe29411c0a0 con 0x7fe294104210 2024-09-11T08:01:54.652 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.642+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe29411aea0 con 0x7fe294104210 2024-09-11T08:01:54.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.642+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 668761296 0 0) 0x7fe29411d2a0 con 0x7fe294107e30 2024-09-11T08:01:54.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.642+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe29411c0a0 con 0x7fe294107e30 2024-09-11T08:01:54.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.642+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fe28c001c00 con 0x7fe294107e30 2024-09-11T08:01:54.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.642+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 550082619 0 0) 0x7fe2800034e0 con 0x7fe2941004b0 2024-09-11T08:01:54.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.642+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe29411d2a0 con 0x7fe2941004b0 2024-09-11T08:01:54.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.642+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 944170387 0 0) 0x7fe29411aea0 con 0x7fe294104210 2024-09-11T08:01:54.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.643+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe2800034e0 con 0x7fe294104210 2024-09-11T08:01:54.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.643+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fe284001880 con 0x7fe2941004b0 2024-09-11T08:01:54.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.643+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fe27c001510 con 0x7fe294104210 2024-09-11T08:01:54.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.643+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2252845526 0 0) 0x7fe29411c0a0 con 0x7fe294107e30 2024-09-11T08:01:54.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.643+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 >> v1:172.21.15.102:6790/0 conn(0x7fe2941004b0 legacy=0x7fe294104780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:54.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.643+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 >> v1:172.21.15.146:6789/0 conn(0x7fe294104210 legacy=0x7fe294104e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:54.653 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.643+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29411e4a0 con 0x7fe294107e30 2024-09-11T08:01:54.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.643+0000 7fe298926640 1 -- 172.21.15.102:0/639186890 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29411b090 con 0x7fe294107e30 2024-09-11T08:01:54.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.643+0000 7fe298926640 1 -- 172.21.15.102:0/639186890 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29411b4e0 con 0x7fe294107e30 2024-09-11T08:01:54.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.643+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7fe28c0023c0 con 0x7fe294107e30 2024-09-11T08:01:54.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.643+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fe28c003620 con 0x7fe294107e30 2024-09-11T08:01:54.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.644+0000 7fe298926640 1 -- 172.21.15.102:0/639186890 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe274005010 con 0x7fe294107e30 2024-09-11T08:01:54.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.644+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7fe28c001f70 con 0x7fe294107e30 2024-09-11T08:01:54.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.645+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 376332686 0 0) 0x7fe28c087930 con 0x7fe294107e30 2024-09-11T08:01:54.654 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.649+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fe28c054200 con 0x7fe294107e30 2024-09-11T08:01:54.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.807+0000 7fe298926640 1 -- 172.21.15.102:0/639186890 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe2740052c0 con 0x7fe294107e30 2024-09-11T08:01:54.809 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.808+0000 7fe28affd640 1 -- 172.21.15.102:0/639186890 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (unknown 383520633 0 2838807938) 0x7fe28c002220 con 0x7fe294107e30 2024-09-11T08:01:54.809 INFO:teuthology.orchestra.run.smithi102.stdout:107374182438 2024-09-11T08:01:54.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.810+0000 7fe298926640 1 -- 172.21.15.102:0/639186890 >> v1:172.21.15.102:6800/4281557748 conn(0x7fe28006fe40 legacy=0x7fe2800722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:54.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.810+0000 7fe298926640 1 -- 172.21.15.102:0/639186890 >> v1:172.21.15.102:6789/0 conn(0x7fe294107e30 legacy=0x7fe2941195a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:54.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.810+0000 7fe298926640 1 -- 172.21.15.102:0/639186890 shutdown_connections 2024-09-11T08:01:54.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.810+0000 7fe298926640 1 -- 172.21.15.102:0/639186890 >> 172.21.15.102:0/639186890 conn(0x7fe294074a50 msgr2=0x7fe294100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:54.811 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.810+0000 7fe298926640 1 -- 172.21.15.102:0/639186890 shutdown_connections 2024-09-11T08:01:54.812 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:54.810+0000 7fe298926640 1 -- 172.21.15.102:0/639186890 wait complete. 2024-09-11T08:01:54.877 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:01:54 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:01:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:01:55.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.314+0000 7f4408b81640 1 -- 172.21.15.102:0/1075264059 >> v1:172.21.15.102:6789/0 conn(0x7f4404105320 legacy=0x7f4404107700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:55.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.314+0000 7f4408b81640 1 -- 172.21.15.102:0/1075264059 shutdown_connections 2024-09-11T08:01:55.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.314+0000 7f4408b81640 1 -- 172.21.15.102:0/1075264059 >> 172.21.15.102:0/1075264059 conn(0x7f44040f8140 msgr2=0x7f44040fa550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:55.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.314+0000 7f4408b81640 1 -- 172.21.15.102:0/1075264059 shutdown_connections 2024-09-11T08:01:55.316 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.314+0000 7f4408b81640 1 -- 172.21.15.102:0/1075264059 wait complete. 2024-09-11T08:01:55.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.315+0000 7f4408b81640 1 Processor -- start 2024-09-11T08:01:55.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.315+0000 7f4408b81640 1 -- start start 2024-09-11T08:01:55.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.315+0000 7f4408b81640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f440411be80 con 0x7f4404102560 2024-09-11T08:01:55.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.315+0000 7f4408b81640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f440411d080 con 0x7f4404108eb0 2024-09-11T08:01:55.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.315+0000 7f4408b81640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f440411e280 con 0x7f4404105320 2024-09-11T08:01:55.318 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.315+0000 7f440259c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f4404102560 0x7f44040fe570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:43478/0 (socket says 172.21.15.102:43478) 2024-09-11T08:01:55.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.315+0000 7f440259c640 1 -- 172.21.15.102:0/2506026894 learned_addr learned my addr 172.21.15.102:0/2506026894 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:01:55.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.316+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3607090405 0 0) 0x7f440411e280 con 0x7f4404105320 2024-09-11T08:01:55.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.316+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43ec0034e0 con 0x7f4404105320 2024-09-11T08:01:55.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.316+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110074873 0 0) 0x7f440411be80 con 0x7f4404102560 2024-09-11T08:01:55.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.316+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f440411e280 con 0x7f4404102560 2024-09-11T08:01:55.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.316+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 652776786 0 0) 0x7f440411d080 con 0x7f4404108eb0 2024-09-11T08:01:55.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.316+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f440411be80 con 0x7f4404108eb0 2024-09-11T08:01:55.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.316+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1705503404 0 0) 0x7f43ec0034e0 con 0x7f4404105320 2024-09-11T08:01:55.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.316+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f440411d080 con 0x7f4404105320 2024-09-11T08:01:55.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.316+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f43fc0016e0 con 0x7f4404105320 2024-09-11T08:01:55.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.316+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2055735350 0 0) 0x7f440411e280 con 0x7f4404102560 2024-09-11T08:01:55.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.316+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f43ec0034e0 con 0x7f4404102560 2024-09-11T08:01:55.319 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.316+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f43f4001880 con 0x7f4404102560 2024-09-11T08:01:55.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.316+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3533202503 0 0) 0x7f440411be80 con 0x7f4404108eb0 2024-09-11T08:01:55.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.316+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f440411e280 con 0x7f4404108eb0 2024-09-11T08:01:55.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.316+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f43f80018d0 con 0x7f4404108eb0 2024-09-11T08:01:55.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.316+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1117888519 0 0) 0x7f440411d080 con 0x7f4404105320 2024-09-11T08:01:55.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.316+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 >> v1:172.21.15.146:6789/0 conn(0x7f4404108eb0 legacy=0x7f440411a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:55.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.316+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 >> v1:172.21.15.102:6789/0 conn(0x7f4404102560 legacy=0x7f44040fe570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:55.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.316+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440411f480 con 0x7f4404105320 2024-09-11T08:01:55.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.316+0000 7f4408b81640 1 -- 172.21.15.102:0/2506026894 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f440411e470 con 0x7f4404105320 2024-09-11T08:01:55.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.316+0000 7f4408b81640 1 -- 172.21.15.102:0/2506026894 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f440411e940 con 0x7f4404105320 2024-09-11T08:01:55.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.316+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f43fc002fc0 con 0x7f4404105320 2024-09-11T08:01:55.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.316+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f43fc003820 con 0x7f4404105320 2024-09-11T08:01:55.320 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.317+0000 7f4408b81640 1 -- 172.21.15.102:0/2506026894 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e4005010 con 0x7f4404105320 2024-09-11T08:01:55.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.318+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7f43fc002400 con 0x7f4404105320 2024-09-11T08:01:55.321 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.318+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 376332686 0 0) 0x7f43fc087bc0 con 0x7f4404105320 2024-09-11T08:01:55.323 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.322+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f43fc054910 con 0x7f4404105320 2024-09-11T08:01:55.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.483+0000 7f4408b81640 1 -- 172.21.15.102:0/2506026894 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f43e40052c0 con 0x7f4404105320 2024-09-11T08:01:55.484 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.483+0000 7f43f2ffd640 1 -- 172.21.15.102:0/2506026894 <== mon.2 v1:172.21.15.102:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (unknown 2131975755 0 1574476285) 0x7f43fc0542b0 con 0x7f4404105320 2024-09-11T08:01:55.484 INFO:teuthology.orchestra.run.smithi102.stdout:154618822686 2024-09-11T08:01:55.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.485+0000 7f4408b81640 1 -- 172.21.15.102:0/2506026894 >> v1:172.21.15.102:6800/4281557748 conn(0x7f43ec06fe60 legacy=0x7f43ec072310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:55.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.485+0000 7f4408b81640 1 -- 172.21.15.102:0/2506026894 >> v1:172.21.15.102:6790/0 conn(0x7f4404105320 legacy=0x7f44040fec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:55.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.485+0000 7f4408b81640 1 -- 172.21.15.102:0/2506026894 shutdown_connections 2024-09-11T08:01:55.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.485+0000 7f4408b81640 1 -- 172.21.15.102:0/2506026894 >> 172.21.15.102:0/2506026894 conn(0x7f44040f8140 msgr2=0x7f440410c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:55.486 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.485+0000 7f4408b81640 1 -- 172.21.15.102:0/2506026894 shutdown_connections 2024-09-11T08:01:55.487 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:55.485+0000 7f4408b81640 1 -- 172.21.15.102:0/2506026894 wait complete. 2024-09-11T08:01:56.055 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:55 smithi102 ceph-mon[41371]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:56.055 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/639186890' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T08:01:56.055 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2506026894' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T08:01:56.055 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:55 smithi102 ceph-mon[29849]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:56.055 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/639186890' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T08:01:56.056 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2506026894' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T08:01:56.079 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822683 got 154618822686 for osd.5 2024-09-11T08:01:56.079 DEBUG:teuthology.parallel:result is None 2024-09-11T08:01:56.139 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:55 smithi146 ceph-mon[33158]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:56.139 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/639186890' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T08:01:56.139 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2506026894' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T08:01:56.446 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182434 got 107374182438 for osd.3 2024-09-11T08:01:56.446 DEBUG:teuthology.parallel:result is None 2024-09-11T08:01:56.447 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T08:01:56.447 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:01:56.693 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:01:56.938 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:01:56 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[76080]: ts=2024-09-11T08:01:56.673Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.004183592s 2024-09-11T08:01:57.931 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:57 smithi102 ceph-mon[41371]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:01:57.931 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:57 smithi102 ceph-mon[29849]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:01:57.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:57 smithi146 ceph-mon[33158]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:01:58.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.125+0000 7f5340b89640 1 -- 172.21.15.102:0/3251346488 >> v1:172.21.15.102:6789/0 conn(0x7f533c101530 legacy=0x7f533c101900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:58.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.125+0000 7f5340b89640 1 -- 172.21.15.102:0/3251346488 shutdown_connections 2024-09-11T08:01:58.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.125+0000 7f5340b89640 1 -- 172.21.15.102:0/3251346488 >> 172.21.15.102:0/3251346488 conn(0x7f533c0752c0 msgr2=0x7f533c0f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:58.132 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.125+0000 7f5340b89640 1 -- 172.21.15.102:0/3251346488 shutdown_connections 2024-09-11T08:01:58.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.125+0000 7f5340b89640 1 -- 172.21.15.102:0/3251346488 wait complete. 2024-09-11T08:01:58.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.125+0000 7f5340b89640 1 Processor -- start 2024-09-11T08:01:58.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.125+0000 7f5340b89640 1 -- start start 2024-09-11T08:01:58.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f5340b89640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f533c11ac90 con 0x7f533c101530 2024-09-11T08:01:58.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f5340b89640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f533c11be50 con 0x7f533c1042a0 2024-09-11T08:01:58.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f5340b89640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f533c11d050 con 0x7f533c107e00 2024-09-11T08:01:58.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f533a59c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f533c101530 0x7f533c104670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:55176/0 (socket says 172.21.15.102:55176) 2024-09-11T08:01:58.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f533a59c640 1 -- 172.21.15.102:0/786223335 learned_addr learned my addr 172.21.15.102:0/786223335 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:01:58.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f533ad9d640 1 --1- 172.21.15.102:0/786223335 >> v1:172.21.15.102:6790/0 conn(0x7f533c107e00 0x7f533c119450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:51516/0 (socket says 172.21.15.102:51516) 2024-09-11T08:01:58.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 595540769 0 0) 0x7f533c11d050 con 0x7f533c107e00 2024-09-11T08:01:58.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53240034e0 con 0x7f533c107e00 2024-09-11T08:01:58.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2452807059 0 0) 0x7f533c11ac90 con 0x7f533c101530 2024-09-11T08:01:58.133 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f533c11d050 con 0x7f533c101530 2024-09-11T08:01:58.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123795731 0 0) 0x7f533c11be50 con 0x7f533c1042a0 2024-09-11T08:01:58.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f533c11ac90 con 0x7f533c1042a0 2024-09-11T08:01:58.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2798714663 0 0) 0x7f53240034e0 con 0x7f533c107e00 2024-09-11T08:01:58.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f533c11be50 con 0x7f533c107e00 2024-09-11T08:01:58.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f532c002b40 con 0x7f533c107e00 2024-09-11T08:01:58.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2792420682 0 0) 0x7f533c11d050 con 0x7f533c101530 2024-09-11T08:01:58.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f53240034e0 con 0x7f533c101530 2024-09-11T08:01:58.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f5334002a20 con 0x7f533c101530 2024-09-11T08:01:58.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1164652220 0 0) 0x7f533c11ac90 con 0x7f533c1042a0 2024-09-11T08:01:58.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f533c11d050 con 0x7f533c1042a0 2024-09-11T08:01:58.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f53300016a0 con 0x7f533c1042a0 2024-09-11T08:01:58.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1836135017 0 0) 0x7f533c11be50 con 0x7f533c107e00 2024-09-11T08:01:58.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 >> v1:172.21.15.146:6789/0 conn(0x7f533c1042a0 legacy=0x7f533c104d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:58.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 >> v1:172.21.15.102:6789/0 conn(0x7f533c101530 legacy=0x7f533c104670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:58.134 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533c11e250 con 0x7f533c107e00 2024-09-11T08:01:58.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.126+0000 7f5340b89640 1 -- 172.21.15.102:0/786223335 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f533c11ae80 con 0x7f533c107e00 2024-09-11T08:01:58.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.127+0000 7f5340b89640 1 -- 172.21.15.102:0/786223335 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f533c11b350 con 0x7f533c107e00 2024-09-11T08:01:58.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.127+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f532c0030d0 con 0x7f533c107e00 2024-09-11T08:01:58.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.127+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f532c005ca0 con 0x7f533c107e00 2024-09-11T08:01:58.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.128+0000 7f5340b89640 1 -- 172.21.15.102:0/786223335 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531c005010 con 0x7f533c107e00 2024-09-11T08:01:58.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.128+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7f532c001250 con 0x7f533c107e00 2024-09-11T08:01:58.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.128+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 376332686 0 0) 0x7f532c089de0 con 0x7f533c107e00 2024-09-11T08:01:58.135 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.132+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f532c056ad0 con 0x7f533c107e00 2024-09-11T08:01:58.285 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.284+0000 7f5340b89640 1 -- 172.21.15.102:0/786223335 --> v1:172.21.15.102:6800/4281557748 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f531c000b90 con 0x7f532406fe40 2024-09-11T08:01:58.297 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.296+0000 7f532affd640 1 -- 172.21.15.102:0/786223335 <== mgr.24497 v1:172.21.15.102:6800/4281557748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+383802 (unknown 2965378022 0 3980644532) 0x7f531c000b90 con 0x7f532406fe40 2024-09-11T08:01:58.301 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T08:01:58.303 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-11T08:01:58.306 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 podman[66633]: 2024-09-11 08:01:58.18833258 +0000 UTC m=+0.127473466 container create 1dd8b360aa501f9edd30fd0b7f0894060e546f3e7054a6af2b9b4b9e368a8d44 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a, name=ubi8, build-date=2022-04-29T11:48:11.055897, com.redhat.component=ubi8-container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., architecture=x86_64, io.openshift.tags=base rhel8, vcs-type=git, vendor=Red Hat, Inc., version=8.5, io.openshift.expose-services=, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, description=Ceph Grafana Container, distribution-scope=public, maintainer=Paul Cuzner , summary=Grafana Container configured for Ceph mgr/dashboard integration, io.buildah.version=1.32.0, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=Red Hat Universal Base Image 8, release=239.1651231664) 2024-09-11T08:01:58.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.300+0000 7f5340b89640 1 -- 172.21.15.102:0/786223335 >> v1:172.21.15.102:6800/4281557748 conn(0x7f532406fe40 legacy=0x7f53240722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:58.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.300+0000 7f5340b89640 1 -- 172.21.15.102:0/786223335 >> v1:172.21.15.102:6790/0 conn(0x7f533c107e00 legacy=0x7f533c119450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:01:58.306 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.300+0000 7f5340b89640 1 -- 172.21.15.102:0/786223335 shutdown_connections 2024-09-11T08:01:58.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.300+0000 7f5340b89640 1 -- 172.21.15.102:0/786223335 >> 172.21.15.102:0/786223335 conn(0x7f533c0752c0 msgr2=0x7f533c10b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:01:58.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.300+0000 7f5340b89640 1 -- 172.21.15.102:0/786223335 shutdown_connections 2024-09-11T08:01:58.307 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:01:58.300+0000 7f5340b89640 1 -- 172.21.15.102:0/786223335 wait complete. 2024-09-11T08:01:58.667 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 podman[66633]: 2024-09-11 08:01:58.415890752 +0000 UTC m=+0.355031640 container init 1dd8b360aa501f9edd30fd0b7f0894060e546f3e7054a6af2b9b4b9e368a8d44 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a, io.buildah.version=1.32.0, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vendor=Red Hat, Inc., name=ubi8, version=8.5, io.k8s.display-name=Red Hat Universal Base Image 8, description=Ceph Grafana Container, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vcs-type=git, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, build-date=2022-04-29T11:48:11.055897, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.openshift.expose-services=, com.redhat.component=ubi8-container, distribution-scope=public, architecture=x86_64, io.openshift.tags=base rhel8, maintainer=Paul Cuzner , release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56) 2024-09-11T08:01:58.667 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 podman[66633]: 2024-09-11 08:01:58.420848619 +0000 UTC m=+0.359989504 container start 1dd8b360aa501f9edd30fd0b7f0894060e546f3e7054a6af2b9b4b9e368a8d44 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=, vendor=Red Hat, Inc., io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.component=ubi8-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., description=Ceph Grafana Container, distribution-scope=public, maintainer=Paul Cuzner , release=239.1651231664, vcs-type=git, name=ubi8, build-date=2022-04-29T11:48:11.055897, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.buildah.version=1.32.0, version=8.5, architecture=x86_64, io.openshift.tags=base rhel8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56) 2024-09-11T08:01:58.667 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 bash[66633]: 1dd8b360aa501f9edd30fd0b7f0894060e546f3e7054a6af2b9b4b9e368a8d44 2024-09-11T08:01:58.667 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 systemd[1]: Started Ceph grafana.a for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T08:01:58.667 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=settings t=2024-09-11T08:01:58.584680514Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-11T08:01:58.667 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=settings t=2024-09-11T08:01:58.584875155Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-11T08:01:58.667 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=settings t=2024-09-11T08:01:58.584894381Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-11T08:01:58.668 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=settings t=2024-09-11T08:01:58.584902171Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-11T08:01:58.668 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=settings t=2024-09-11T08:01:58.584908634Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-11T08:01:58.668 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=settings t=2024-09-11T08:01:58.584914709Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-11T08:01:58.668 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=settings t=2024-09-11T08:01:58.584920602Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-11T08:01:58.668 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=settings t=2024-09-11T08:01:58.584926518Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-11T08:01:58.668 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=settings t=2024-09-11T08:01:58.58493237Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-11T08:01:58.668 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=settings t=2024-09-11T08:01:58.584938193Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-11T08:01:58.668 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=settings t=2024-09-11T08:01:58.584943827Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-11T08:01:58.668 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=settings t=2024-09-11T08:01:58.584949212Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-11T08:01:58.668 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=settings t=2024-09-11T08:01:58.584955622Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-11T08:01:58.668 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=settings t=2024-09-11T08:01:58.584961514Z level=info msg="App mode production" 2024-09-11T08:01:58.668 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=sqlstore t=2024-09-11T08:01:58.585012058Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-11T08:01:58.669 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=sqlstore t=2024-09-11T08:01:58.585029726Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-11T08:01:58.669 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.585896426Z level=info msg="Starting DB migrations" 2024-09-11T08:01:58.669 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.585974113Z level=info msg="Executing migration" id="create migration_log table" 2024-09-11T08:01:58.669 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.58650495Z level=info msg="Executing migration" id="create user table" 2024-09-11T08:01:58.669 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.587114404Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-11T08:01:58.669 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.589939813Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-11T08:01:58.669 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.592354567Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-11T08:01:58.669 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.595234396Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-11T08:01:58.670 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.595671801Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-11T08:01:58.670 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.598586988Z level=info msg="Executing migration" id="create user table v2" 2024-09-11T08:01:58.670 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.599065883Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-11T08:01:58.670 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.60179502Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-11T08:01:58.670 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.604715796Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-11T08:01:58.670 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.607380598Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-11T08:01:58.670 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.610134816Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-11T08:01:58.670 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.612912628Z level=info msg="Executing migration" id="Update user table charset" 2024-09-11T08:01:58.670 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.615478653Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-11T08:01:58.670 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.618338791Z level=info msg="Executing migration" id="Add missing user data" 2024-09-11T08:01:58.671 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.62108183Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-11T08:01:58.671 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.623698068Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-11T08:01:58.671 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.626382195Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-11T08:01:58.671 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.629133639Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-11T08:01:58.671 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.631650926Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-11T08:01:58.671 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.634529624Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-11T08:01:58.671 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.636901759Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-11T08:01:58.671 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.637333634Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-11T08:01:58.671 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.640189401Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-11T08:01:58.671 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.642922422Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-11T08:01:58.671 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.645414018Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-11T08:01:58.671 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.64825788Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-11T08:01:58.672 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.650925043Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-11T08:01:58.672 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.651322934Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-11T08:01:58.672 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.652131178Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-11T08:01:58.672 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.652994712Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-11T08:01:58.672 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.653731021Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-11T08:01:58.672 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.654190128Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-11T08:01:58.672 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.654981535Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-11T08:01:58.672 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.655886806Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-11T08:01:58.672 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.656590431Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-11T08:01:58.672 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.65695258Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-11T08:01:58.672 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.657851545Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-11T08:01:58.672 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.658325031Z level=info msg="Executing migration" id="create star table" 2024-09-11T08:01:58.673 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.658707746Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-11T08:01:58.673 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.659487604Z level=info msg="Executing migration" id="create org table v1" 2024-09-11T08:01:58.673 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.660330969Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-11T08:01:58.673 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.661169948Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-11T08:01:58.673 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.662004234Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-11T08:01:58.673 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.662956774Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-11T08:01:58.673 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.663674702Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-11T08:01:58.673 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.664648417Z level=info msg="Executing migration" id="Update org table charset" 2024-09-11T08:01:58.673 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.665379266Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-11T08:01:58.673 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.666423796Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-11T08:01:58.673 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.667075796Z level=info msg="Executing migration" id="create dashboard table" 2024-09-11T08:01:58.673 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.667923693Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-11T08:01:58.918 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.66878904Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-11T08:01:58.918 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.669324442Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-11T08:01:58.919 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.670182361Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-11T08:01:58.919 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.671017488Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-11T08:01:58.919 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.671936514Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-11T08:01:58.919 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.674663039Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-11T08:01:58.919 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.677564175Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-11T08:01:58.920 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.680293051Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-11T08:01:58.920 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.683301514Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-11T08:01:58.920 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.686039974Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-11T08:01:58.920 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.688766069Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-11T08:01:58.920 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.691760998Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-11T08:01:58.920 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.69450119Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-11T08:01:58.920 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.697095836Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-11T08:01:58.921 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.70002657Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-11T08:01:58.921 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.702898076Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-11T08:01:58.921 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.705731577Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-11T08:01:58.921 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.708439351Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-11T08:01:58.921 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.711155001Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-11T08:01:58.921 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.713955219Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-11T08:01:58.921 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.716788244Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-11T08:01:58.922 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.719520138Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-11T08:01:58.922 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.720095739Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-11T08:01:58.922 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.722968016Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-11T08:01:58.922 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.725587246Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-11T08:01:58.922 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.728314382Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-11T08:01:58.922 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.73070937Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-11T08:01:58.922 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.73353658Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-11T08:01:58.922 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.735966115Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-11T08:01:58.922 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.738609748Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-11T08:01:58.922 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.74134524Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-11T08:01:58.922 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.743965891Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-11T08:01:58.922 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.74651496Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-11T08:01:58.922 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.74923719Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-11T08:01:58.922 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.751937929Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-11T08:01:58.923 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.754656194Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-11T08:01:58.923 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.757294329Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-11T08:01:58.923 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.759892443Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-11T08:01:58.924 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.762303314Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-11T08:01:58.924 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.764850305Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-11T08:01:58.924 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.767592192Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-11T08:01:58.924 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.768469082Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-11T08:01:58.924 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.769207091Z level=info msg="Executing migration" id="create data_source table" 2024-09-11T08:01:58.924 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.770055673Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-11T08:01:58.924 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.770832875Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-11T08:01:58.925 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.771300112Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-11T08:01:58.925 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.772123957Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-11T08:01:58.925 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.773073748Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-11T08:01:58.925 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.774152028Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-11T08:01:58.925 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.775136365Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-11T08:01:58.925 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.775842783Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-11T08:01:58.925 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.776683091Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-11T08:01:58.925 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.77751592Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-11T08:01:58.925 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.778349953Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-11T08:01:58.925 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.779270804Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-11T08:01:58.925 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.780137832Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-11T08:01:58.925 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.780858002Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-11T08:01:58.925 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.781835098Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-11T08:01:58.926 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.782505435Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-11T08:01:58.926 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.783443292Z level=info msg="Executing migration" id="Add uid column" 2024-09-11T08:01:58.926 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.78419843Z level=info msg="Executing migration" id="Update uid value" 2024-09-11T08:01:58.926 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.785192521Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-11T08:01:58.926 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.785907527Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-11T08:01:58.926 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.786945666Z level=info msg="Executing migration" id="create api_key table" 2024-09-11T08:01:58.926 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.787670846Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-11T08:01:58.926 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.788502055Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-11T08:01:58.926 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.789355043Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-11T08:01:58.926 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.790288953Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-11T08:01:58.927 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.791195766Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-11T08:01:58.927 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.792074924Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-11T08:01:58.927 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.792821673Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-11T08:01:58.927 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.793874504Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-11T08:01:58.927 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.794673374Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-11T08:01:58.927 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.795090671Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-11T08:01:58.927 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.795520789Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-11T08:01:58.927 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.795942001Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-11T08:01:58.927 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.796253932Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-11T08:01:58.927 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.79661804Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-11T08:01:58.927 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.796890167Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-11T08:01:58.928 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.797472263Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-11T08:01:58.928 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.798018261Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-11T08:01:58.928 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.798316222Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-11T08:01:58.928 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.79886161Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-11T08:01:58.928 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.799416502Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-11T08:01:58.928 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.799819485Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-11T08:01:58.928 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.800159361Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-11T08:01:58.928 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.800575646Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-11T08:01:58.928 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.80099391Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-11T08:01:58.928 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.801424598Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-11T08:01:58.928 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.801844491Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-11T08:01:58.928 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.802126329Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-11T08:01:58.928 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.802448798Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-11T08:01:58.929 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.803036034Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-11T08:01:58.929 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.803631635Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-11T08:01:58.929 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.803956571Z level=info msg="Executing migration" id="create quota table v1" 2024-09-11T08:01:58.929 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.804351495Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-11T08:01:58.929 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.804787688Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-11T08:01:58.929 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.805060872Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-11T08:01:58.929 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.805479057Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-11T08:01:58.929 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.805923051Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-11T08:01:58.929 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.806537877Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-11T08:01:58.929 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.806849001Z level=info msg="Executing migration" id="create session table" 2024-09-11T08:01:58.929 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.807291039Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-11T08:01:58.929 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.807603108Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-11T08:01:58.929 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.807887628Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-11T08:01:58.930 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.80829224Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-11T08:01:58.930 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.80869037Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-11T08:01:58.930 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.808956489Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-11T08:01:58.930 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.809227187Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-11T08:01:58.930 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.8095497Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-11T08:01:58.930 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.809886345Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-11T08:01:58.930 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.810455679Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-11T08:01:58.930 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.810765981Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-11T08:01:58.930 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.811387412Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-11T08:01:58.930 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.81173998Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-11T08:01:58.930 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.812305444Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-11T08:01:58.930 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.812908513Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-11T08:01:58.930 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.813183651Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-11T08:01:58.931 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.813622265Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-11T08:01:58.931 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.81406986Z level=info msg="Executing migration" id="create alert table v1" 2024-09-11T08:01:58.931 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.814880915Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-11T08:01:58.931 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.815752102Z level=info msg="Executing migration" id="add index alert state" 2024-09-11T08:01:58.931 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.8161818Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-11T08:01:58.931 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.816661911Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-11T08:01:58.931 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.817097326Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-11T08:01:58.931 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.817537176Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-11T08:01:58.931 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.817972892Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-11T08:01:58.931 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.819458241Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-11T08:01:58.931 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.819849294Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-11T08:01:58.931 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.820306787Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-11T08:01:58.931 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.820644003Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-11T08:01:58.932 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.821045189Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-11T08:01:58.932 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.821494714Z level=info msg="Executing migration" id="Add column is_default" 2024-09-11T08:01:58.932 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.822144756Z level=info msg="Executing migration" id="Add column frequency" 2024-09-11T08:01:58.932 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.822800963Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-11T08:01:58.932 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.823453268Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-11T08:01:58.932 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.82410739Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-11T08:01:58.932 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.824598779Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-11T08:01:58.932 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.824924406Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-11T08:01:58.932 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.825204032Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-11T08:01:58.932 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.825651876Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-11T08:01:58.932 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.826194076Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-11T08:01:58.932 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.827181203Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-11T08:01:58.932 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.827894132Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-11T08:01:58.932 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.828784031Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-11T08:01:58.933 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.829631199Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-11T08:01:58.933 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.830294836Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-11T08:01:58.933 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.831150273Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-11T08:01:58.933 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.832116101Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-11T08:01:58.933 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.832959899Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-11T08:01:58.933 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.833825649Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-11T08:01:58.933 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.834723737Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-11T08:01:58.933 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.837669333Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-11T08:01:58.933 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.840374589Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-11T08:01:58.933 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.840707984Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-11T08:01:58.933 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.843589971Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-11T08:01:58.933 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.846528951Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-11T08:01:58.933 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.849355641Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-11T08:01:58.934 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.849823269Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-11T08:01:58.934 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.852578639Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-11T08:01:58.934 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.855482352Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-11T08:01:58.934 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.858318332Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-11T08:01:58.934 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.859017886Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-11T08:01:58.934 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.862021945Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-11T08:01:58.934 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.864802704Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-11T08:01:58.934 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.867409174Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-11T08:01:58.934 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.867844542Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-11T08:01:58.934 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.87059214Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-11T08:01:58.934 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.872186266Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-11T08:01:58.934 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.873455931Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-11T08:01:58.934 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.876450513Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-11T08:01:58.934 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.876780497Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-11T08:01:58.934 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.879502404Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-11T08:01:58.935 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.879830299Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-11T08:01:58.935 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.882208881Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-11T08:01:58.935 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.885039209Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-11T08:01:58.935 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.887734571Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-11T08:01:58.935 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.890265528Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-11T08:01:58.935 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.892967586Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-11T08:01:58.935 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.89573027Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-11T08:01:58.935 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.898489318Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-11T08:01:58.935 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.901108676Z level=info msg="Executing migration" id="Move region to single row" 2024-09-11T08:01:58.935 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.903966636Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-11T08:01:58.935 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.906597116Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-11T08:01:58.935 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.909357169Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-11T08:01:58.935 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.911917291Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-11T08:01:58.936 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.91485557Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-11T08:01:58.936 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.915530118Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-11T08:01:58.936 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.916312778Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-11T08:01:58.936 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.917214674Z level=info msg="Executing migration" id="create test_data table" 2024-09-11T08:01:58.936 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.917935397Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-11T08:01:58.936 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.918746586Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-11T08:01:58.936 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.919572157Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-11T08:01:58.936 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.92035503Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-11T08:01:58.936 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.921140269Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-11T08:01:58.936 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.921932947Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-11T08:01:58.952 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-11T08:01:56.338147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":924,"num_read_kb":781,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385380,"kb_used_data":6544,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575836,"statfs":{"total":767960285184,"available":765517656064,"internally_reserved":0,"allocated":6701056,"data_stored":3631507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002960"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222044+0000","last_change":"2024-09-11T08:00:43.071118+0000","last_active":"2024-09-11T08:00:46.222044+0000","last_peered":"2024-09-11T08:00:46.222044+0000","last_clean":"2024-09-11T08:00:46.222044+0000","last_became_active":"2024-09-11T08:00:43.070866+0000","last_became_peered":"2024-09-11T08:00:43.070866+0000","last_unstale":"2024-09-11T08:00:46.222044+0000","last_undegraded":"2024-09-11T08:00:46.222044+0000","last_fullsized":"2024-09-11T08:00:46.222044+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:04.189160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.145008+0000","last_change":"2024-09-11T08:00:36.806461+0000","last_active":"2024-09-11T08:00:46.145008+0000","last_peered":"2024-09-11T08:00:46.145008+0000","last_clean":"2024-09-11T08:00:46.145008+0000","last_became_active":"2024-09-11T08:00:36.806166+0000","last_became_peered":"2024-09-11T08:00:36.806166+0000","last_unstale":"2024-09-11T08:00:46.145008+0000","last_undegraded":"2024-09-11T08:00:46.145008+0000","last_fullsized":"2024-09-11T08:00:46.145008+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:08.181846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222351+0000","last_change":"2024-09-11T08:00:38.821552+0000","last_active":"2024-09-11T08:00:46.222351+0000","last_peered":"2024-09-11T08:00:46.222351+0000","last_clean":"2024-09-11T08:00:46.222351+0000","last_became_active":"2024-09-11T08:00:38.821284+0000","last_became_peered":"2024-09-11T08:00:38.821284+0000","last_unstale":"2024-09-11T08:00:46.222351+0000","last_undegraded":"2024-09-11T08:00:46.222351+0000","last_fullsized":"2024-09-11T08:00:46.222351+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:29:25.936260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.141575+0000","last_change":"2024-09-11T08:00:40.833382+0000","last_active":"2024-09-11T08:00:46.141575+0000","last_peered":"2024-09-11T08:00:46.141575+0000","last_clean":"2024-09-11T08:00:46.141575+0000","last_became_active":"2024-09-11T08:00:40.833096+0000","last_became_peered":"2024-09-11T08:00:40.833096+0000","last_unstale":"2024-09-11T08:00:46.141575+0000","last_undegraded":"2024-09-11T08:00:46.141575+0000","last_fullsized":"2024-09-11T08:00:46.141575+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:33:53.440815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222758+0000","last_change":"2024-09-11T08:00:36.807450+0000","last_active":"2024-09-11T08:00:46.222758+0000","last_peered":"2024-09-11T08:00:46.222758+0000","last_clean":"2024-09-11T08:00:46.222758+0000","last_became_active":"2024-09-11T08:00:36.807201+0000","last_became_peered":"2024-09-11T08:00:36.807201+0000","last_unstale":"2024-09-11T08:00:46.222758+0000","last_undegraded":"2024-09-11T08:00:46.222758+0000","last_fullsized":"2024-09-11T08:00:46.222758+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:34:19.120603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.141584+0000","last_change":"2024-09-11T08:00:38.807270+0000","last_active":"2024-09-11T08:00:46.141584+0000","last_peered":"2024-09-11T08:00:46.141584+0000","last_clean":"2024-09-11T08:00:46.141584+0000","last_became_active":"2024-09-11T08:00:38.806941+0000","last_became_peered":"2024-09-11T08:00:38.806941+0000","last_unstale":"2024-09-11T08:00:46.141584+0000","last_undegraded":"2024-09-11T08:00:46.141584+0000","last_fullsized":"2024-09-11T08:00:46.141584+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:18:26.615405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143144+0000","last_change":"2024-09-11T08:00:40.835014+0000","last_active":"2024-09-11T08:00:46.143144+0000","last_peered":"2024-09-11T08:00:46.143144+0000","last_clean":"2024-09-11T08:00:46.143144+0000","last_became_active":"2024-09-11T08:00:40.834514+0000","last_became_peered":"2024-09-11T08:00:40.834514+0000","last_unstale":"2024-09-11T08:00:46.143144+0000","last_undegraded":"2024-09-11T08:00:46.143144+0000","last_fullsized":"2024-09-11T08:00:46.143144+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:41:53.217112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.141805+0000","last_change":"2024-09-11T08:00:42.830798+0000","last_active":"2024-09-11T08:00:46.141805+0000","last_peered":"2024-09-11T08:00:46.141805+0000","last_clean":"2024-09-11T08:00:46.141805+0000","last_became_active":"2024-09-11T08:00:42.830511+0000","last_became_peered":"2024-09-11T08:00:42.830511+0000","last_unstale":"2024-09-11T08:00:46.141805+0000","last_undegraded":"2024-09-11T08:00:46.141805+0000","last_fullsized":"2024-09-11T08:00:46.141805+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:47:47.676397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227650+0000","last_change":"2024-09-11T08:00:36.809729+0000","last_active":"2024-09-11T08:00:46.227650+0000","last_peered":"2024-09-11T08:00:46.227650+0000","last_clean":"2024-09-11T08:00:46.227650+0000","last_became_active":"2024-09-11T08:00:36.809471+0000","last_became_peered":"2024-09-11T08:00:36.809471+0000","last_unstale":"2024-09-11T08:00:46.227650+0000","last_undegraded":"2024-09-11T08:00:46.227650+0000","last_fullsized":"2024-09-11T08:00:46.227650+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:58.420627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"59'5","reported_seq":26,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.141896+0000","last_change":"2024-09-11T08:00:38.814013+0000","last_active":"2024-09-11T08:00:46.141896+0000","last_peered":"2024-09-11T08:00:46.141896+0000","last_clean":"2024-09-11T08:00:46.141896+0000","last_became_active":"2024-09-11T08:00:38.813580+0000","last_became_peered":"2024-09-11T08:00:38.813580+0000","last_unstale":"2024-09-11T08:00:46.141896+0000","last_undegraded":"2024-09-11T08:00:46.141896+0000","last_fullsized":"2024-09-11T08:00:46.141896+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:37:34.146632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227002+0000","last_change":"2024-09-11T08:00:40.835066+0000","last_active":"2024-09-11T08:00:46.227002+0000","last_peered":"2024-09-11T08:00:46.227002+0000","last_clean":"2024-09-11T08:00:46.227002+0000","last_became_active":"2024-09-11T08:00:40.833127+0000","last_became_peered":"2024-09-11T08:00:40.833127+0000","last_unstale":"2024-09-11T08:00:46.227002+0000","last_undegraded":"2024-09-11T08:00:46.227002+0000","last_fullsized":"2024-09-11T08:00:46.227002+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:02:23.340171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226723+0000","last_change":"2024-09-11T08:00:42.831044+0000","last_active":"2024-09-11T08:00:46.226723+0000","last_peered":"2024-09-11T08:00:46.226723+0000","last_clean":"2024-09-11T08:00:46.226723+0000","last_became_active":"2024-09-11T08:00:42.830649+0000","last_became_peered":"2024-09-11T08:00:42.830649+0000","last_unstale":"2024-09-11T08:00:46.226723+0000","last_undegraded":"2024-09-11T08:00:46.226723+0000","last_fullsized":"2024-09-11T08:00:46.226723+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:15:35.987319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227737+0000","last_change":"2024-09-11T08:00:36.802253+0000","last_active":"2024-09-11T08:00:46.227737+0000","last_peered":"2024-09-11T08:00:46.227737+0000","last_clean":"2024-09-11T08:00:46.227737+0000","last_became_active":"2024-09-11T08:00:36.801488+0000","last_became_peered":"2024-09-11T08:00:36.801488+0000","last_unstale":"2024-09-11T08:00:46.227737+0000","last_undegraded":"2024-09-11T08:00:46.227737+0000","last_fullsized":"2024-09-11T08:00:46.227737+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:36:37.539262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.141921+0000","last_change":"2024-09-11T08:00:38.817584+0000","last_active":"2024-09-11T08:00:46.141921+0000","last_peered":"2024-09-11T08:00:46.141921+0000","last_clean":"2024-09-11T08:00:46.141921+0000","last_became_active":"2024-09-11T08:00:38.817293+0000","last_became_peered":"2024-09-11T08:00:38.817293+0000","last_unstale":"2024-09-11T08:00:46.141921+0000","last_undegraded":"2024-09-11T08:00:46.141921+0000","last_fullsized":"2024-09-11T08:00:46.141921+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:07:27.412741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226809+0000","last_change":"2024-09-11T08:00:40.840328+0000","last_active":"2024-09-11T08:00:46.226809+0000","last_peered":"2024-09-11T08:00:46.226809+0000","last_clean":"2024-09-11T08:00:46.226809+0000","last_became_active":"2024-09-11T08:00:40.840015+0000","last_became_peered":"2024-09-11T08:00:40.840015+0000","last_unstale":"2024-09-11T08:00:46.226809+0000","last_undegraded":"2024-09-11T08:00:46.226809+0000","last_fullsized":"2024-09-11T08:00:46.226809+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:13:51.451498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143377+0000","last_change":"2024-09-11T08:00:42.834881+0000","last_active":"2024-09-11T08:00:46.143377+0000","last_peered":"2024-09-11T08:00:46.143377+0000","last_clean":"2024-09-11T08:00:46.143377+0000","last_became_active":"2024-09-11T08:00:42.834526+0000","last_became_peered":"2024-09-11T08:00:42.834526+0000","last_unstale":"2024-09-11T08:00:46.143377+0000","last_undegraded":"2024-09-11T08:00:46.143377+0000","last_fullsized":"2024-09-11T08:00:46.143377+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:56:20.510162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"59'17","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.142268+0000","last_change":"2024-09-11T08:00:38.810541+0000","last_active":"2024-09-11T08:00:46.142268+0000","last_peered":"2024-09-11T08:00:46.142268+0000","last_clean":"2024-09-11T08:00:46.142268+0000","last_became_active":"2024-09-11T08:00:38.810233+0000","last_became_peered":"2024-09-11T08:00:38.810233+0000","last_unstale":"2024-09-11T08:00:46.142268+0000","last_undegraded":"2024-09-11T08:00:46.142268+0000","last_fullsized":"2024-09-11T08:00:46.142268+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:01:30.198295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227468+0000","last_change":"2024-09-11T08:00:36.802119+0000","last_active":"2024-09-11T08:00:46.227468+0000","last_peered":"2024-09-11T08:00:46.227468+0000","last_clean":"2024-09-11T08:00:46.227468+0000","last_became_active":"2024-09-11T08:00:36.801499+0000","last_became_peered":"2024-09-11T08:00:36.801499+0000","last_unstale":"2024-09-11T08:00:46.227468+0000","last_undegraded":"2024-09-11T08:00:46.227468+0000","last_fullsized":"2024-09-11T08:00:46.227468+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:19:04.266476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.145179+0000","last_change":"2024-09-11T08:00:40.835188+0000","last_active":"2024-09-11T08:00:46.145179+0000","last_peered":"2024-09-11T08:00:46.145179+0000","last_clean":"2024-09-11T08:00:46.145179+0000","last_became_active":"2024-09-11T08:00:40.834922+0000","last_became_peered":"2024-09-11T08:00:40.834922+0000","last_unstale":"2024-09-11T08:00:46.145179+0000","last_undegraded":"2024-09-11T08:00:46.145179+0000","last_fullsized":"2024-09-11T08:00:46.145179+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:07:44.376883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227296+0000","last_change":"2024-09-11T08:00:42.835330+0000","last_active":"2024-09-11T08:00:46.227296+0000","last_peered":"2024-09-11T08:00:46.227296+0000","last_clean":"2024-09-11T08:00:46.227296+0000","last_became_active":"2024-09-11T08:00:42.835044+0000","last_became_peered":"2024-09-11T08:00:42.835044+0000","last_unstale":"2024-09-11T08:00:46.227296+0000","last_undegraded":"2024-09-11T08:00:46.227296+0000","last_fullsized":"2024-09-11T08:00:46.227296+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:56:19.084364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.142085+0000","last_change":"2024-09-11T08:00:38.813670+0000","last_active":"2024-09-11T08:00:46.142085+0000","last_peered":"2024-09-11T08:00:46.142085+0000","last_clean":"2024-09-11T08:00:46.142085+0000","last_became_active":"2024-09-11T08:00:38.813399+0000","last_became_peered":"2024-09-11T08:00:38.813399+0000","last_unstale":"2024-09-11T08:00:46.142085+0000","last_undegraded":"2024-09-11T08:00:46.142085+0000","last_fullsized":"2024-09-11T08:00:46.142085+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:28:56.757990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144882+0000","last_change":"2024-09-11T08:00:36.799756+0000","last_active":"2024-09-11T08:00:46.144882+0000","last_peered":"2024-09-11T08:00:46.144882+0000","last_clean":"2024-09-11T08:00:46.144882+0000","last_became_active":"2024-09-11T08:00:36.799313+0000","last_became_peered":"2024-09-11T08:00:36.799313+0000","last_unstale":"2024-09-11T08:00:46.144882+0000","last_undegraded":"2024-09-11T08:00:46.144882+0000","last_fullsized":"2024-09-11T08:00:46.144882+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:40:43.725089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:01:50.144238+0000","last_change":"2024-09-11T08:00:40.834916+0000","last_active":"2024-09-11T08:01:50.144238+0000","last_peered":"2024-09-11T08:01:50.144238+0000","last_clean":"2024-09-11T08:01:50.144238+0000","last_became_active":"2024-09-11T08:00:40.834572+0000","last_became_peered":"2024-09-11T08:00:40.834572+0000","last_unstale":"2024-09-11T08:01:50.144238+0000","last_undegraded":"2024-09-11T08:01:50.144238+0000","last_fullsized":"2024-09-11T08:01:50.144238+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:07:51.608546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.142422+0000","last_change":"2024-09-11T08:00:42.831080+0000","last_active":"2024-09-11T08:00:46.142422+0000","last_peered":"2024-09-11T08:00:46.142422+0000","last_clean":"2024-09-11T08:00:46.142422+0000","last_became_active":"2024-09-11T08:00:42.830716+0000","last_became_peered":"2024-09-11T08:00:42.830716+0000","last_unstale":"2024-09-11T08:00:46.142422+0000","last_undegraded":"2024-09-11T08:00:46.142422+0000","last_fullsized":"2024-09-11T08:00:46.142422+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:32:53.539884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144421+0000","last_change":"2024-09-11T08:00:38.816566+0000","last_active":"2024-09-11T08:00:46.144421+0000","last_peered":"2024-09-11T08:00:46.144421+0000","last_clean":"2024-09-11T08:00:46.144421+0000","last_became_active":"2024-09-11T08:00:38.816286+0000","last_became_peered":"2024-09-11T08:00:38.816286+0000","last_unstale":"2024-09-11T08:00:46.144421+0000","last_undegraded":"2024-09-11T08:00:46.144421+0000","last_fullsized":"2024-09-11T08:00:46.144421+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:07:00.156358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143336+0000","last_change":"2024-09-11T08:00:36.799352+0000","last_active":"2024-09-11T08:00:46.143336+0000","last_peered":"2024-09-11T08:00:46.143336+0000","last_clean":"2024-09-11T08:00:46.143336+0000","last_became_active":"2024-09-11T08:00:36.798975+0000","last_became_peered":"2024-09-11T08:00:36.798975+0000","last_unstale":"2024-09-11T08:00:46.143336+0000","last_undegraded":"2024-09-11T08:00:46.143336+0000","last_fullsized":"2024-09-11T08:00:46.143336+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:47:23.251644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:01:50.144535+0000","last_change":"2024-09-11T08:00:40.829051+0000","last_active":"2024-09-11T08:01:50.144535+0000","last_peered":"2024-09-11T08:01:50.144535+0000","last_clean":"2024-09-11T08:01:50.144535+0000","last_became_active":"2024-09-11T08:00:40.828790+0000","last_became_peered":"2024-09-11T08:00:40.828790+0000","last_unstale":"2024-09-11T08:01:50.144535+0000","last_undegraded":"2024-09-11T08:01:50.144535+0000","last_fullsized":"2024-09-11T08:01:50.144535+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:44:00.386495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.228188+0000","last_change":"2024-09-11T08:00:42.832968+0000","last_active":"2024-09-11T08:00:46.228188+0000","last_peered":"2024-09-11T08:00:46.228188+0000","last_clean":"2024-09-11T08:00:46.228188+0000","last_became_active":"2024-09-11T08:00:42.832644+0000","last_became_peered":"2024-09-11T08:00:42.832644+0000","last_unstale":"2024-09-11T08:00:46.228188+0000","last_undegraded":"2024-09-11T08:00:46.228188+0000","last_fullsized":"2024-09-11T08:00:46.228188+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:49.382900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.142506+0000","last_change":"2024-09-11T08:00:38.807986+0000","last_active":"2024-09-11T08:00:46.142506+0000","last_peered":"2024-09-11T08:00:46.142506+0000","last_clean":"2024-09-11T08:00:46.142506+0000","last_became_active":"2024-09-11T08:00:38.807500+0000","last_became_peered":"2024-09-11T08:00:38.807500+0000","last_unstale":"2024-09-11T08:00:46.142506+0000","last_undegraded":"2024-09-11T08:00:46.142506+0000","last_fullsized":"2024-09-11T08:00:46.142506+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:37:38.524178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226722+0000","last_change":"2024-09-11T08:00:36.809196+0000","last_active":"2024-09-11T08:00:46.226722+0000","last_peered":"2024-09-11T08:00:46.226722+0000","last_clean":"2024-09-11T08:00:46.226722+0000","last_became_active":"2024-09-11T08:00:36.808925+0000","last_became_peered":"2024-09-11T08:00:36.808925+0000","last_unstale":"2024-09-11T08:00:46.226722+0000","last_undegraded":"2024-09-11T08:00:46.226722+0000","last_fullsized":"2024-09-11T08:00:46.226722+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:40:06.627847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.228016+0000","last_change":"2024-09-11T08:00:40.832397+0000","last_active":"2024-09-11T08:00:46.228016+0000","last_peered":"2024-09-11T08:00:46.228016+0000","last_clean":"2024-09-11T08:00:46.228016+0000","last_became_active":"2024-09-11T08:00:40.832005+0000","last_became_peered":"2024-09-11T08:00:40.832005+0000","last_unstale":"2024-09-11T08:00:46.228016+0000","last_undegraded":"2024-09-11T08:00:46.228016+0000","last_fullsized":"2024-09-11T08:00:46.228016+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:44:41.009589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222270+0000","last_change":"2024-09-11T08:00:43.068306+0000","last_active":"2024-09-11T08:00:46.222270+0000","last_peered":"2024-09-11T08:00:46.222270+0000","last_clean":"2024-09-11T08:00:46.222270+0000","last_became_active":"2024-09-11T08:00:43.067787+0000","last_became_peered":"2024-09-11T08:00:43.067787+0000","last_unstale":"2024-09-11T08:00:46.222270+0000","last_undegraded":"2024-09-11T08:00:46.222270+0000","last_fullsized":"2024-09-11T08:00:46.222270+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:52.034174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"59'14","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.141657+0000","last_change":"2024-09-11T08:00:38.808225+0000","last_active":"2024-09-11T08:00:46.141657+0000","last_peered":"2024-09-11T08:00:46.141657+0000","last_clean":"2024-09-11T08:00:46.141657+0000","last_became_active":"2024-09-11T08:00:38.807792+0000","last_became_peered":"2024-09-11T08:00:38.807792+0000","last_unstale":"2024-09-11T08:00:46.141657+0000","last_undegraded":"2024-09-11T08:00:46.141657+0000","last_fullsized":"2024-09-11T08:00:46.141657+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:46:04.937830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"52'1","reported_seq":27,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.200735+0000","last_change":"2024-09-11T08:00:36.799406+0000","last_active":"2024-09-11T08:00:46.200735+0000","last_peered":"2024-09-11T08:00:46.200735+0000","last_clean":"2024-09-11T08:00:46.200735+0000","last_became_active":"2024-09-11T08:00:36.799038+0000","last_became_peered":"2024-09-11T08:00:36.799038+0000","last_unstale":"2024-09-11T08:00:46.200735+0000","last_undegraded":"2024-09-11T08:00:46.200735+0000","last_fullsized":"2024-09-11T08:00:46.200735+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:11:05.226318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:01:50.144381+0000","last_change":"2024-09-11T08:00:40.834172+0000","last_active":"2024-09-11T08:01:50.144381+0000","last_peered":"2024-09-11T08:01:50.144381+0000","last_clean":"2024-09-11T08:01:50.144381+0000","last_became_active":"2024-09-11T08:00:40.833864+0000","last_became_peered":"2024-09-11T08:00:40.833864+0000","last_unstale":"2024-09-11T08:01:50.144381+0000","last_undegraded":"2024-09-11T08:01:50.144381+0000","last_fullsized":"2024-09-11T08:01:50.144381+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:46:10.088736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227285+0000","last_change":"2024-09-11T08:00:43.070101+0000","last_active":"2024-09-11T08:00:46.227285+0000","last_peered":"2024-09-11T08:00:46.227285+0000","last_clean":"2024-09-11T08:00:46.227285+0000","last_became_active":"2024-09-11T08:00:43.069799+0000","last_became_peered":"2024-09-11T08:00:43.069799+0000","last_unstale":"2024-09-11T08:00:46.227285+0000","last_undegraded":"2024-09-11T08:00:46.227285+0000","last_fullsized":"2024-09-11T08:00:46.227285+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:42:56.848971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"59'18","reported_seq":48,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.221973+0000","last_change":"2024-09-11T08:00:39.123505+0000","last_active":"2024-09-11T08:00:46.221973+0000","last_peered":"2024-09-11T08:00:46.221973+0000","last_clean":"2024-09-11T08:00:46.221973+0000","last_became_active":"2024-09-11T08:00:39.123126+0000","last_became_peered":"2024-09-11T08:00:39.123126+0000","last_unstale":"2024-09-11T08:00:46.221973+0000","last_undegraded":"2024-09-11T08:00:46.221973+0000","last_fullsized":"2024-09-11T08:00:46.221973+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:13:50.011710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144754+0000","last_change":"2024-09-11T08:00:36.796896+0000","last_active":"2024-09-11T08:00:46.144754+0000","last_peered":"2024-09-11T08:00:46.144754+0000","last_clean":"2024-09-11T08:00:46.144754+0000","last_became_active":"2024-09-11T08:00:36.796626+0000","last_became_peered":"2024-09-11T08:00:36.796626+0000","last_unstale":"2024-09-11T08:00:46.144754+0000","last_undegraded":"2024-09-11T08:00:46.144754+0000","last_fullsized":"2024-09-11T08:00:46.144754+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:57:03.740594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.141575+0000","last_change":"2024-09-11T08:00:40.831495+0000","last_active":"2024-09-11T08:00:46.141575+0000","last_peered":"2024-09-11T08:00:46.141575+0000","last_clean":"2024-09-11T08:00:46.141575+0000","last_became_active":"2024-09-11T08:00:40.831201+0000","last_became_peered":"2024-09-11T08:00:40.831201+0000","last_unstale":"2024-09-11T08:00:46.141575+0000","last_undegraded":"2024-09-11T08:00:46.141575+0000","last_fullsized":"2024-09-11T08:00:46.141575+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:40.593930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.141954+0000","last_change":"2024-09-11T08:00:42.834169+0000","last_active":"2024-09-11T08:00:46.141954+0000","last_peered":"2024-09-11T08:00:46.141954+0000","last_clean":"2024-09-11T08:00:46.141954+0000","last_became_active":"2024-09-11T08:00:42.833883+0000","last_became_peered":"2024-09-11T08:00:42.833883+0000","last_unstale":"2024-09-11T08:00:46.141954+0000","last_undegraded":"2024-09-11T08:00:46.141954+0000","last_fullsized":"2024-09-11T08:00:46.141954+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:22:37.209284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"59'19","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144705+0000","last_change":"2024-09-11T08:00:39.123210+0000","last_active":"2024-09-11T08:00:46.144705+0000","last_peered":"2024-09-11T08:00:46.144705+0000","last_clean":"2024-09-11T08:00:46.144705+0000","last_became_active":"2024-09-11T08:00:39.122933+0000","last_became_peered":"2024-09-11T08:00:39.122933+0000","last_unstale":"2024-09-11T08:00:46.144705+0000","last_undegraded":"2024-09-11T08:00:46.144705+0000","last_fullsized":"2024-09-11T08:00:46.144705+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:09:07.029660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227720+0000","last_change":"2024-09-11T08:00:36.809169+0000","last_active":"2024-09-11T08:00:46.227720+0000","last_peered":"2024-09-11T08:00:46.227720+0000","last_clean":"2024-09-11T08:00:46.227720+0000","last_became_active":"2024-09-11T08:00:36.808895+0000","last_became_peered":"2024-09-11T08:00:36.808895+0000","last_unstale":"2024-09-11T08:00:46.227720+0000","last_undegraded":"2024-09-11T08:00:46.227720+0000","last_fullsized":"2024-09-11T08:00:46.227720+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:39:10.932536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.200272+0000","last_change":"2024-09-11T08:00:40.839308+0000","last_active":"2024-09-11T08:00:46.200272+0000","last_peered":"2024-09-11T08:00:46.200272+0000","last_clean":"2024-09-11T08:00:46.200272+0000","last_became_active":"2024-09-11T08:00:40.839007+0000","last_became_peered":"2024-09-11T08:00:40.839007+0000","last_unstale":"2024-09-11T08:00:46.200272+0000","last_undegraded":"2024-09-11T08:00:46.200272+0000","last_fullsized":"2024-09-11T08:00:46.200272+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:49:28.124681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143065+0000","last_change":"2024-09-11T08:00:43.068167+0000","last_active":"2024-09-11T08:00:46.143065+0000","last_peered":"2024-09-11T08:00:46.143065+0000","last_clean":"2024-09-11T08:00:46.143065+0000","last_became_active":"2024-09-11T08:00:43.067709+0000","last_became_peered":"2024-09-11T08:00:43.067709+0000","last_unstale":"2024-09-11T08:00:46.143065+0000","last_undegraded":"2024-09-11T08:00:46.143065+0000","last_fullsized":"2024-09-11T08:00:46.143065+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:22:51.854452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144724+0000","last_change":"2024-09-11T08:00:38.808521+0000","last_active":"2024-09-11T08:00:46.144724+0000","last_peered":"2024-09-11T08:00:46.144724+0000","last_clean":"2024-09-11T08:00:46.144724+0000","last_became_active":"2024-09-11T08:00:38.807979+0000","last_became_peered":"2024-09-11T08:00:38.807979+0000","last_unstale":"2024-09-11T08:00:46.144724+0000","last_undegraded":"2024-09-11T08:00:46.144724+0000","last_fullsized":"2024-09-11T08:00:46.144724+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:32:57.633567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.145321+0000","last_change":"2024-09-11T08:00:36.805492+0000","last_active":"2024-09-11T08:00:46.145321+0000","last_peered":"2024-09-11T08:00:46.145321+0000","last_clean":"2024-09-11T08:00:46.145321+0000","last_became_active":"2024-09-11T08:00:36.805216+0000","last_became_peered":"2024-09-11T08:00:36.805216+0000","last_unstale":"2024-09-11T08:00:46.145321+0000","last_undegraded":"2024-09-11T08:00:46.145321+0000","last_fullsized":"2024-09-11T08:00:46.145321+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:33:03.369780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:01:50.144240+0000","last_change":"2024-09-11T08:00:40.840792+0000","last_active":"2024-09-11T08:01:50.144240+0000","last_peered":"2024-09-11T08:01:50.144240+0000","last_clean":"2024-09-11T08:01:50.144240+0000","last_became_active":"2024-09-11T08:00:40.840525+0000","last_became_peered":"2024-09-11T08:00:40.840525+0000","last_unstale":"2024-09-11T08:01:50.144240+0000","last_undegraded":"2024-09-11T08:01:50.144240+0000","last_fullsized":"2024-09-11T08:01:50.144240+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:41:58.906478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143158+0000","last_change":"2024-09-11T08:00:42.838006+0000","last_active":"2024-09-11T08:00:46.143158+0000","last_peered":"2024-09-11T08:00:46.143158+0000","last_clean":"2024-09-11T08:00:46.143158+0000","last_became_active":"2024-09-11T08:00:42.837727+0000","last_became_peered":"2024-09-11T08:00:42.837727+0000","last_unstale":"2024-09-11T08:00:46.143158+0000","last_undegraded":"2024-09-11T08:00:46.143158+0000","last_fullsized":"2024-09-11T08:00:46.143158+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:58:16.999026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"59'28","reported_seq":80,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:01:50.144495+0000","last_change":"2024-09-11T08:00:38.819419+0000","last_active":"2024-09-11T08:01:50.144495+0000","last_peered":"2024-09-11T08:01:50.144495+0000","last_clean":"2024-09-11T08:01:50.144495+0000","last_became_active":"2024-09-11T08:00:38.818939+0000","last_became_peered":"2024-09-11T08:00:38.818939+0000","last_unstale":"2024-09-11T08:01:50.144495+0000","last_undegraded":"2024-09-11T08:01:50.144495+0000","last_fullsized":"2024-09-11T08:01:50.144495+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:00:23.229784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227189+0000","last_change":"2024-09-11T08:00:36.799345+0000","last_active":"2024-09-11T08:00:46.227189+0000","last_peered":"2024-09-11T08:00:46.227189+0000","last_clean":"2024-09-11T08:00:46.227189+0000","last_became_active":"2024-09-11T08:00:36.799090+0000","last_became_peered":"2024-09-11T08:00:36.799090+0000","last_unstale":"2024-09-11T08:00:46.227189+0000","last_undegraded":"2024-09-11T08:00:46.227189+0000","last_fullsized":"2024-09-11T08:00:46.227189+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:57:00.554977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143668+0000","last_change":"2024-09-11T08:00:40.837484+0000","last_active":"2024-09-11T08:00:46.143668+0000","last_peered":"2024-09-11T08:00:46.143668+0000","last_clean":"2024-09-11T08:00:46.143668+0000","last_became_active":"2024-09-11T08:00:40.837112+0000","last_became_peered":"2024-09-11T08:00:40.837112+0000","last_unstale":"2024-09-11T08:00:46.143668+0000","last_undegraded":"2024-09-11T08:00:46.143668+0000","last_fullsized":"2024-09-11T08:00:46.143668+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:50:31.274414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222628+0000","last_change":"2024-09-11T08:00:42.831942+0000","last_active":"2024-09-11T08:00:46.222628+0000","last_peered":"2024-09-11T08:00:46.222628+0000","last_clean":"2024-09-11T08:00:46.222628+0000","last_became_active":"2024-09-11T08:00:42.831669+0000","last_became_peered":"2024-09-11T08:00:42.831669+0000","last_unstale":"2024-09-11T08:00:46.222628+0000","last_undegraded":"2024-09-11T08:00:46.222628+0000","last_fullsized":"2024-09-11T08:00:46.222628+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:09:59.680384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"59'13","reported_seq":43,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144565+0000","last_change":"2024-09-11T08:00:38.817893+0000","last_active":"2024-09-11T08:00:46.144565+0000","last_peered":"2024-09-11T08:00:46.144565+0000","last_clean":"2024-09-11T08:00:46.144565+0000","last_became_active":"2024-09-11T08:00:38.817550+0000","last_became_peered":"2024-09-11T08:00:38.817550+0000","last_unstale":"2024-09-11T08:00:46.144565+0000","last_undegraded":"2024-09-11T08:00:46.144565+0000","last_fullsized":"2024-09-11T08:00:46.144565+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:40:28.158387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"60'5","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:01:53.394683+0000","last_change":"2024-09-11T08:00:40.823639+0000","last_active":"2024-09-11T08:01:53.394683+0000","last_peered":"2024-09-11T08:01:53.394683+0000","last_clean":"2024-09-11T08:01:53.394683+0000","last_became_active":"2024-09-11T08:00:36.796900+0000","last_became_peered":"2024-09-11T08:00:36.796900+0000","last_unstale":"2024-09-11T08:01:53.394683+0000","last_undegraded":"2024-09-11T08:01:53.394683+0000","last_fullsized":"2024-09-11T08:01:53.394683+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:26:47.119529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039650379999999997,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":71,"num_read_kb":66,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226928+0000","last_change":"2024-09-11T08:00:36.806792+0000","last_active":"2024-09-11T08:00:46.226928+0000","last_peered":"2024-09-11T08:00:46.226928+0000","last_clean":"2024-09-11T08:00:46.226928+0000","last_became_active":"2024-09-11T08:00:36.806365+0000","last_became_peered":"2024-09-11T08:00:36.806365+0000","last_unstale":"2024-09-11T08:00:46.226928+0000","last_undegraded":"2024-09-11T08:00:46.226928+0000","last_fullsized":"2024-09-11T08:00:46.226928+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:39:38.024552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227292+0000","last_change":"2024-09-11T08:00:40.825697+0000","last_active":"2024-09-11T08:00:46.227292+0000","last_peered":"2024-09-11T08:00:46.227292+0000","last_clean":"2024-09-11T08:00:46.227292+0000","last_became_active":"2024-09-11T08:00:40.825391+0000","last_became_peered":"2024-09-11T08:00:40.825391+0000","last_unstale":"2024-09-11T08:00:46.227292+0000","last_undegraded":"2024-09-11T08:00:46.227292+0000","last_fullsized":"2024-09-11T08:00:46.227292+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:19:41.889688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227383+0000","last_change":"2024-09-11T08:00:43.068381+0000","last_active":"2024-09-11T08:00:46.227383+0000","last_peered":"2024-09-11T08:00:46.227383+0000","last_clean":"2024-09-11T08:00:46.227383+0000","last_became_active":"2024-09-11T08:00:43.068092+0000","last_became_peered":"2024-09-11T08:00:43.068092+0000","last_unstale":"2024-09-11T08:00:46.227383+0000","last_undegraded":"2024-09-11T08:00:46.227383+0000","last_fullsized":"2024-09-11T08:00:46.227383+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:01.390541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"59'12","reported_seq":34,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144838+0000","last_change":"2024-09-11T08:00:38.820878+0000","last_active":"2024-09-11T08:00:46.144838+0000","last_peered":"2024-09-11T08:00:46.144838+0000","last_clean":"2024-09-11T08:00:46.144838+0000","last_became_active":"2024-09-11T08:00:38.820561+0000","last_became_peered":"2024-09-11T08:00:38.820561+0000","last_unstale":"2024-09-11T08:00:46.144838+0000","last_undegraded":"2024-09-11T08:00:46.144838+0000","last_fullsized":"2024-09-11T08:00:46.144838+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:43:02.244072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"54'1","reported_seq":24,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144162+0000","last_change":"2024-09-11T08:00:40.826552+0000","last_active":"2024-09-11T08:00:46.144162+0000","last_peered":"2024-09-11T08:00:46.144162+0000","last_clean":"2024-09-11T08:00:46.144162+0000","last_became_active":"2024-09-11T08:00:36.806294+0000","last_became_peered":"2024-09-11T08:00:36.806294+0000","last_unstale":"2024-09-11T08:00:46.144162+0000","last_undegraded":"2024-09-11T08:00:46.144162+0000","last_fullsized":"2024-09-11T08:00:46.144162+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:38:39.602628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061749899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226758+0000","last_change":"2024-09-11T08:00:36.809229+0000","last_active":"2024-09-11T08:00:46.226758+0000","last_peered":"2024-09-11T08:00:46.226758+0000","last_clean":"2024-09-11T08:00:46.226758+0000","last_became_active":"2024-09-11T08:00:36.808915+0000","last_became_peered":"2024-09-11T08:00:36.808915+0000","last_unstale":"2024-09-11T08:00:46.226758+0000","last_undegraded":"2024-09-11T08:00:46.226758+0000","last_fullsized":"2024-09-11T08:00:46.226758+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:42:22.872114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.228060+0000","last_change":"2024-09-11T08:00:40.839326+0000","last_active":"2024-09-11T08:00:46.228060+0000","last_peered":"2024-09-11T08:00:46.228060+0000","last_clean":"2024-09-11T08:00:46.228060+0000","last_became_active":"2024-09-11T08:00:40.838821+0000","last_became_peered":"2024-09-11T08:00:40.838821+0000","last_unstale":"2024-09-11T08:00:46.228060+0000","last_undegraded":"2024-09-11T08:00:46.228060+0000","last_fullsized":"2024-09-11T08:00:46.228060+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:27:19.392496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.145096+0000","last_change":"2024-09-11T08:00:42.830924+0000","last_active":"2024-09-11T08:00:46.145096+0000","last_peered":"2024-09-11T08:00:46.145096+0000","last_clean":"2024-09-11T08:00:46.145096+0000","last_became_active":"2024-09-11T08:00:42.830587+0000","last_became_peered":"2024-09-11T08:00:42.830587+0000","last_unstale":"2024-09-11T08:00:46.145096+0000","last_undegraded":"2024-09-11T08:00:46.145096+0000","last_fullsized":"2024-09-11T08:00:46.145096+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:10:34.296714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"59'14","reported_seq":38,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.200423+0000","last_change":"2024-09-11T08:00:39.123611+0000","last_active":"2024-09-11T08:00:46.200423+0000","last_peered":"2024-09-11T08:00:46.200423+0000","last_clean":"2024-09-11T08:00:46.200423+0000","last_became_active":"2024-09-11T08:00:39.123207+0000","last_became_peered":"2024-09-11T08:00:39.123207+0000","last_unstale":"2024-09-11T08:00:46.200423+0000","last_undegraded":"2024-09-11T08:00:46.200423+0000","last_fullsized":"2024-09-11T08:00:46.200423+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:08:50.532757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'2","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222219+0000","last_change":"2024-09-11T08:00:40.826568+0000","last_active":"2024-09-11T08:00:46.222219+0000","last_peered":"2024-09-11T08:00:46.222219+0000","last_clean":"2024-09-11T08:00:46.222219+0000","last_became_active":"2024-09-11T08:00:36.797923+0000","last_became_peered":"2024-09-11T08:00:36.797923+0000","last_unstale":"2024-09-11T08:00:46.222219+0000","last_undegraded":"2024-09-11T08:00:46.222219+0000","last_fullsized":"2024-09-11T08:00:46.222219+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:09:52.150614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028090279999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227911+0000","last_change":"2024-09-11T08:00:36.801632+0000","last_active":"2024-09-11T08:00:46.227911+0000","last_peered":"2024-09-11T08:00:46.227911+0000","last_clean":"2024-09-11T08:00:46.227911+0000","last_became_active":"2024-09-11T08:00:36.801375+0000","last_became_peered":"2024-09-11T08:00:36.801375+0000","last_unstale":"2024-09-11T08:00:46.227911+0000","last_undegraded":"2024-09-11T08:00:46.227911+0000","last_fullsized":"2024-09-11T08:00:46.227911+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:25:26.929458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"61'79","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:01:16.379109+0000","last_change":"2024-09-11T08:00:25.551556+0000","last_active":"2024-09-11T08:01:16.379109+0000","last_peered":"2024-09-11T08:01:16.379109+0000","last_clean":"2024-09-11T08:01:16.379109+0000","last_became_active":"2024-09-11T08:00:25.547109+0000","last_became_peered":"2024-09-11T08:00:25.547109+0000","last_unstale":"2024-09-11T08:01:16.379109+0000","last_undegraded":"2024-09-11T08:01:16.379109+0000","last_fullsized":"2024-09-11T08:01:16.379109+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:58:30.437120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:58:30.437120+0000","last_clean_scrub_stamp":"2024-09-11T07:58:30.437120+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:18.497543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227796+0000","last_change":"2024-09-11T08:00:40.831304+0000","last_active":"2024-09-11T08:00:46.227796+0000","last_peered":"2024-09-11T08:00:46.227796+0000","last_clean":"2024-09-11T08:00:46.227796+0000","last_became_active":"2024-09-11T08:00:40.831015+0000","last_became_peered":"2024-09-11T08:00:40.831015+0000","last_unstale":"2024-09-11T08:00:46.227796+0000","last_undegraded":"2024-09-11T08:00:46.227796+0000","last_fullsized":"2024-09-11T08:00:46.227796+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:48.050004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.228130+0000","last_change":"2024-09-11T08:00:42.832223+0000","last_active":"2024-09-11T08:00:46.228130+0000","last_peered":"2024-09-11T08:00:46.228130+0000","last_clean":"2024-09-11T08:00:46.228130+0000","last_became_active":"2024-09-11T08:00:42.831908+0000","last_became_peered":"2024-09-11T08:00:42.831908+0000","last_unstale":"2024-09-11T08:00:46.228130+0000","last_undegraded":"2024-09-11T08:00:46.228130+0000","last_fullsized":"2024-09-11T08:00:46.228130+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:13:19.334798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"59'19","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.200503+0000","last_change":"2024-09-11T08:00:39.123880+0000","last_active":"2024-09-11T08:00:46.200503+0000","last_peered":"2024-09-11T08:00:46.200503+0000","last_clean":"2024-09-11T08:00:46.200503+0000","last_became_active":"2024-09-11T08:00:39.123540+0000","last_became_peered":"2024-09-11T08:00:39.123540+0000","last_unstale":"2024-09-11T08:00:46.200503+0000","last_undegraded":"2024-09-11T08:00:46.200503+0000","last_fullsized":"2024-09-11T08:00:46.200503+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:55:10.411428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227377+0000","last_change":"2024-09-11T08:00:36.803359+0000","last_active":"2024-09-11T08:00:46.227377+0000","last_peered":"2024-09-11T08:00:46.227377+0000","last_clean":"2024-09-11T08:00:46.227377+0000","last_became_active":"2024-09-11T08:00:36.803101+0000","last_became_peered":"2024-09-11T08:00:36.803101+0000","last_unstale":"2024-09-11T08:00:46.227377+0000","last_undegraded":"2024-09-11T08:00:46.227377+0000","last_fullsized":"2024-09-11T08:00:46.227377+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:28:45.051668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227023+0000","last_change":"2024-09-11T08:00:40.830973+0000","last_active":"2024-09-11T08:00:46.227023+0000","last_peered":"2024-09-11T08:00:46.227023+0000","last_clean":"2024-09-11T08:00:46.227023+0000","last_became_active":"2024-09-11T08:00:40.830672+0000","last_became_peered":"2024-09-11T08:00:40.830672+0000","last_unstale":"2024-09-11T08:00:46.227023+0000","last_undegraded":"2024-09-11T08:00:46.227023+0000","last_fullsized":"2024-09-11T08:00:46.227023+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:49:06.396532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226795+0000","last_change":"2024-09-11T08:00:42.834584+0000","last_active":"2024-09-11T08:00:46.226795+0000","last_peered":"2024-09-11T08:00:46.226795+0000","last_clean":"2024-09-11T08:00:46.226795+0000","last_became_active":"2024-09-11T08:00:42.834299+0000","last_became_peered":"2024-09-11T08:00:42.834299+0000","last_unstale":"2024-09-11T08:00:46.226795+0000","last_undegraded":"2024-09-11T08:00:46.226795+0000","last_fullsized":"2024-09-11T08:00:46.226795+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:26:51.062757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144569+0000","last_change":"2024-09-11T08:00:38.819310+0000","last_active":"2024-09-11T08:00:46.144569+0000","last_peered":"2024-09-11T08:00:46.144569+0000","last_clean":"2024-09-11T08:00:46.144569+0000","last_became_active":"2024-09-11T08:00:38.818756+0000","last_became_peered":"2024-09-11T08:00:38.818756+0000","last_unstale":"2024-09-11T08:00:46.144569+0000","last_undegraded":"2024-09-11T08:00:46.144569+0000","last_fullsized":"2024-09-11T08:00:46.144569+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:22:22.696900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.145099+0000","last_change":"2024-09-11T08:00:36.797701+0000","last_active":"2024-09-11T08:00:46.145099+0000","last_peered":"2024-09-11T08:00:46.145099+0000","last_clean":"2024-09-11T08:00:46.145099+0000","last_became_active":"2024-09-11T08:00:36.797454+0000","last_became_peered":"2024-09-11T08:00:36.797454+0000","last_unstale":"2024-09-11T08:00:46.145099+0000","last_undegraded":"2024-09-11T08:00:46.145099+0000","last_fullsized":"2024-09-11T08:00:46.145099+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:18:53.934285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226928+0000","last_change":"2024-09-11T08:00:40.833591+0000","last_active":"2024-09-11T08:00:46.226928+0000","last_peered":"2024-09-11T08:00:46.226928+0000","last_clean":"2024-09-11T08:00:46.226928+0000","last_became_active":"2024-09-11T08:00:40.833225+0000","last_became_peered":"2024-09-11T08:00:40.833225+0000","last_unstale":"2024-09-11T08:00:46.226928+0000","last_undegraded":"2024-09-11T08:00:46.226928+0000","last_fullsized":"2024-09-11T08:00:46.226928+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:10:49.193831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144003+0000","last_change":"2024-09-11T08:00:42.833928+0000","last_active":"2024-09-11T08:00:46.144003+0000","last_peered":"2024-09-11T08:00:46.144003+0000","last_clean":"2024-09-11T08:00:46.144003+0000","last_became_active":"2024-09-11T08:00:42.833641+0000","last_became_peered":"2024-09-11T08:00:42.833641+0000","last_unstale":"2024-09-11T08:00:46.144003+0000","last_undegraded":"2024-09-11T08:00:46.144003+0000","last_fullsized":"2024-09-11T08:00:46.144003+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:24:33.720118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226927+0000","last_change":"2024-09-11T08:00:39.123656+0000","last_active":"2024-09-11T08:00:46.226927+0000","last_peered":"2024-09-11T08:00:46.226927+0000","last_clean":"2024-09-11T08:00:46.226927+0000","last_became_active":"2024-09-11T08:00:39.123347+0000","last_became_peered":"2024-09-11T08:00:39.123347+0000","last_unstale":"2024-09-11T08:00:46.226927+0000","last_undegraded":"2024-09-11T08:00:46.226927+0000","last_fullsized":"2024-09-11T08:00:46.226927+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:40:44.909896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227399+0000","last_change":"2024-09-11T08:00:36.801797+0000","last_active":"2024-09-11T08:00:46.227399+0000","last_peered":"2024-09-11T08:00:46.227399+0000","last_clean":"2024-09-11T08:00:46.227399+0000","last_became_active":"2024-09-11T08:00:36.801515+0000","last_became_peered":"2024-09-11T08:00:36.801515+0000","last_unstale":"2024-09-11T08:00:46.227399+0000","last_undegraded":"2024-09-11T08:00:46.227399+0000","last_fullsized":"2024-09-11T08:00:46.227399+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:27.022074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"59'8","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:01:50.144424+0000","last_change":"2024-09-11T08:00:40.829863+0000","last_active":"2024-09-11T08:01:50.144424+0000","last_peered":"2024-09-11T08:01:50.144424+0000","last_clean":"2024-09-11T08:01:50.144424+0000","last_became_active":"2024-09-11T08:00:40.829265+0000","last_became_peered":"2024-09-11T08:00:40.829265+0000","last_unstale":"2024-09-11T08:01:50.144424+0000","last_undegraded":"2024-09-11T08:01:50.144424+0000","last_fullsized":"2024-09-11T08:01:50.144424+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:36.438007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226671+0000","last_change":"2024-09-11T08:00:43.068277+0000","last_active":"2024-09-11T08:00:46.226671+0000","last_peered":"2024-09-11T08:00:46.226671+0000","last_clean":"2024-09-11T08:00:46.226671+0000","last_became_active":"2024-09-11T08:00:43.067892+0000","last_became_peered":"2024-09-11T08:00:43.067892+0000","last_unstale":"2024-09-11T08:00:46.226671+0000","last_undegraded":"2024-09-11T08:00:46.226671+0000","last_fullsized":"2024-09-11T08:00:46.226671+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:24:09.535763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.141611+0000","last_change":"2024-09-11T08:00:38.815899+0000","last_active":"2024-09-11T08:00:46.141611+0000","last_peered":"2024-09-11T08:00:46.141611+0000","last_clean":"2024-09-11T08:00:46.141611+0000","last_became_active":"2024-09-11T08:00:38.815498+0000","last_became_peered":"2024-09-11T08:00:38.815498+0000","last_unstale":"2024-09-11T08:00:46.141611+0000","last_undegraded":"2024-09-11T08:00:46.141611+0000","last_fullsized":"2024-09-11T08:00:46.141611+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:50:21.463132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"52'2","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.142502+0000","last_change":"2024-09-11T08:00:36.809569+0000","last_active":"2024-09-11T08:00:46.142502+0000","last_peered":"2024-09-11T08:00:46.142502+0000","last_clean":"2024-09-11T08:00:46.142502+0000","last_became_active":"2024-09-11T08:00:36.809000+0000","last_became_peered":"2024-09-11T08:00:36.809000+0000","last_unstale":"2024-09-11T08:00:46.142502+0000","last_undegraded":"2024-09-11T08:00:46.142502+0000","last_fullsized":"2024-09-11T08:00:46.142502+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:36:42.736820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226864+0000","last_change":"2024-09-11T08:00:40.836191+0000","last_active":"2024-09-11T08:00:46.226864+0000","last_peered":"2024-09-11T08:00:46.226864+0000","last_clean":"2024-09-11T08:00:46.226864+0000","last_became_active":"2024-09-11T08:00:40.835918+0000","last_became_peered":"2024-09-11T08:00:40.835918+0000","last_unstale":"2024-09-11T08:00:46.226864+0000","last_undegraded":"2024-09-11T08:00:46.226864+0000","last_fullsized":"2024-09-11T08:00:46.226864+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:02:13.637003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222170+0000","last_change":"2024-09-11T08:00:42.829063+0000","last_active":"2024-09-11T08:00:46.222170+0000","last_peered":"2024-09-11T08:00:46.222170+0000","last_clean":"2024-09-11T08:00:46.222170+0000","last_became_active":"2024-09-11T08:00:42.828748+0000","last_became_peered":"2024-09-11T08:00:42.828748+0000","last_unstale":"2024-09-11T08:00:46.222170+0000","last_undegraded":"2024-09-11T08:00:46.222170+0000","last_fullsized":"2024-09-11T08:00:46.222170+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:55:07.283890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144763+0000","last_change":"2024-09-11T08:00:39.123200+0000","last_active":"2024-09-11T08:00:46.144763+0000","last_peered":"2024-09-11T08:00:46.144763+0000","last_clean":"2024-09-11T08:00:46.144763+0000","last_became_active":"2024-09-11T08:00:39.122900+0000","last_became_peered":"2024-09-11T08:00:39.122900+0000","last_unstale":"2024-09-11T08:00:46.144763+0000","last_undegraded":"2024-09-11T08:00:46.144763+0000","last_fullsized":"2024-09-11T08:00:46.144763+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:26:51.490458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"52'1","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226944+0000","last_change":"2024-09-11T08:00:36.808502+0000","last_active":"2024-09-11T08:00:46.226944+0000","last_peered":"2024-09-11T08:00:46.226944+0000","last_clean":"2024-09-11T08:00:46.226944+0000","last_became_active":"2024-09-11T08:00:36.808215+0000","last_became_peered":"2024-09-11T08:00:36.808215+0000","last_unstale":"2024-09-11T08:00:46.226944+0000","last_undegraded":"2024-09-11T08:00:46.226944+0000","last_fullsized":"2024-09-11T08:00:46.226944+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:24:49.381951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222774+0000","last_change":"2024-09-11T08:00:40.831998+0000","last_active":"2024-09-11T08:00:46.222774+0000","last_peered":"2024-09-11T08:00:46.222774+0000","last_clean":"2024-09-11T08:00:46.222774+0000","last_became_active":"2024-09-11T08:00:40.831689+0000","last_became_peered":"2024-09-11T08:00:40.831689+0000","last_unstale":"2024-09-11T08:00:46.222774+0000","last_undegraded":"2024-09-11T08:00:46.222774+0000","last_fullsized":"2024-09-11T08:00:46.222774+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:29:51.364092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227066+0000","last_change":"2024-09-11T08:00:42.837761+0000","last_active":"2024-09-11T08:00:46.227066+0000","last_peered":"2024-09-11T08:00:46.227066+0000","last_clean":"2024-09-11T08:00:46.227066+0000","last_became_active":"2024-09-11T08:00:42.837503+0000","last_became_peered":"2024-09-11T08:00:42.837503+0000","last_unstale":"2024-09-11T08:00:46.227066+0000","last_undegraded":"2024-09-11T08:00:46.227066+0000","last_fullsized":"2024-09-11T08:00:46.227066+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:32:09.094037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.200431+0000","last_change":"2024-09-11T08:00:36.797052+0000","last_active":"2024-09-11T08:00:46.200431+0000","last_peered":"2024-09-11T08:00:46.200431+0000","last_clean":"2024-09-11T08:00:46.200431+0000","last_became_active":"2024-09-11T08:00:36.796792+0000","last_became_peered":"2024-09-11T08:00:36.796792+0000","last_unstale":"2024-09-11T08:00:46.200431+0000","last_undegraded":"2024-09-11T08:00:46.200431+0000","last_fullsized":"2024-09-11T08:00:46.200431+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:38:50.114641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"59'6","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222862+0000","last_change":"2024-09-11T08:00:39.123764+0000","last_active":"2024-09-11T08:00:46.222862+0000","last_peered":"2024-09-11T08:00:46.222862+0000","last_clean":"2024-09-11T08:00:46.222862+0000","last_became_active":"2024-09-11T08:00:39.123515+0000","last_became_peered":"2024-09-11T08:00:39.123515+0000","last_unstale":"2024-09-11T08:00:46.222862+0000","last_undegraded":"2024-09-11T08:00:46.222862+0000","last_fullsized":"2024-09-11T08:00:46.222862+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:32:48.067610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227937+0000","last_change":"2024-09-11T08:00:40.838777+0000","last_active":"2024-09-11T08:00:46.227937+0000","last_peered":"2024-09-11T08:00:46.227937+0000","last_clean":"2024-09-11T08:00:46.227937+0000","last_became_active":"2024-09-11T08:00:40.838445+0000","last_became_peered":"2024-09-11T08:00:40.838445+0000","last_unstale":"2024-09-11T08:00:46.227937+0000","last_undegraded":"2024-09-11T08:00:46.227937+0000","last_fullsized":"2024-09-11T08:00:46.227937+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:06:52.443015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227301+0000","last_change":"2024-09-11T08:00:43.068823+0000","last_active":"2024-09-11T08:00:46.227301+0000","last_peered":"2024-09-11T08:00:46.227301+0000","last_clean":"2024-09-11T08:00:46.227301+0000","last_became_active":"2024-09-11T08:00:43.068528+0000","last_became_peered":"2024-09-11T08:00:43.068528+0000","last_unstale":"2024-09-11T08:00:46.227301+0000","last_undegraded":"2024-09-11T08:00:46.227301+0000","last_fullsized":"2024-09-11T08:00:46.227301+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:38:40.760170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222731+0000","last_change":"2024-09-11T08:00:39.123193+0000","last_active":"2024-09-11T08:00:46.222731+0000","last_peered":"2024-09-11T08:00:46.222731+0000","last_clean":"2024-09-11T08:00:46.222731+0000","last_became_active":"2024-09-11T08:00:39.122917+0000","last_became_peered":"2024-09-11T08:00:39.122917+0000","last_unstale":"2024-09-11T08:00:46.222731+0000","last_undegraded":"2024-09-11T08:00:46.222731+0000","last_fullsized":"2024-09-11T08:00:46.222731+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:08:27.508033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227123+0000","last_change":"2024-09-11T08:00:36.802610+0000","last_active":"2024-09-11T08:00:46.227123+0000","last_peered":"2024-09-11T08:00:46.227123+0000","last_clean":"2024-09-11T08:00:46.227123+0000","last_became_active":"2024-09-11T08:00:36.802368+0000","last_became_peered":"2024-09-11T08:00:36.802368+0000","last_unstale":"2024-09-11T08:00:46.227123+0000","last_undegraded":"2024-09-11T08:00:46.227123+0000","last_fullsized":"2024-09-11T08:00:46.227123+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:04:07.662727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:01:50.144390+0000","last_change":"2024-09-11T08:00:40.836077+0000","last_active":"2024-09-11T08:01:50.144390+0000","last_peered":"2024-09-11T08:01:50.144390+0000","last_clean":"2024-09-11T08:01:50.144390+0000","last_became_active":"2024-09-11T08:00:40.835757+0000","last_became_peered":"2024-09-11T08:00:40.835757+0000","last_unstale":"2024-09-11T08:01:50.144390+0000","last_undegraded":"2024-09-11T08:01:50.144390+0000","last_fullsized":"2024-09-11T08:01:50.144390+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:52:10.537609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143163+0000","last_change":"2024-09-11T08:00:42.837104+0000","last_active":"2024-09-11T08:00:46.143163+0000","last_peered":"2024-09-11T08:00:46.143163+0000","last_clean":"2024-09-11T08:00:46.143163+0000","last_became_active":"2024-09-11T08:00:42.836669+0000","last_became_peered":"2024-09-11T08:00:42.836669+0000","last_unstale":"2024-09-11T08:00:46.143163+0000","last_undegraded":"2024-09-11T08:00:46.143163+0000","last_fullsized":"2024-09-11T08:00:46.143163+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:29.048875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.228158+0000","last_change":"2024-09-11T08:00:39.123814+0000","last_active":"2024-09-11T08:00:46.228158+0000","last_peered":"2024-09-11T08:00:46.228158+0000","last_clean":"2024-09-11T08:00:46.228158+0000","last_became_active":"2024-09-11T08:00:39.123464+0000","last_became_peered":"2024-09-11T08:00:39.123464+0000","last_unstale":"2024-09-11T08:00:46.228158+0000","last_undegraded":"2024-09-11T08:00:46.228158+0000","last_fullsized":"2024-09-11T08:00:46.228158+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:33:48.390737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.145203+0000","last_change":"2024-09-11T08:00:36.805490+0000","last_active":"2024-09-11T08:00:46.145203+0000","last_peered":"2024-09-11T08:00:46.145203+0000","last_clean":"2024-09-11T08:00:46.145203+0000","last_became_active":"2024-09-11T08:00:36.804674+0000","last_became_peered":"2024-09-11T08:00:36.804674+0000","last_unstale":"2024-09-11T08:00:46.145203+0000","last_undegraded":"2024-09-11T08:00:46.145203+0000","last_fullsized":"2024-09-11T08:00:46.145203+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:10.808200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:01:50.144469+0000","last_change":"2024-09-11T08:00:40.827676+0000","last_active":"2024-09-11T08:01:50.144469+0000","last_peered":"2024-09-11T08:01:50.144469+0000","last_clean":"2024-09-11T08:01:50.144469+0000","last_became_active":"2024-09-11T08:00:40.827324+0000","last_became_peered":"2024-09-11T08:00:40.827324+0000","last_unstale":"2024-09-11T08:01:50.144469+0000","last_undegraded":"2024-09-11T08:01:50.144469+0000","last_fullsized":"2024-09-11T08:01:50.144469+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:07:00.275085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.142278+0000","last_change":"2024-09-11T08:00:42.829729+0000","last_active":"2024-09-11T08:00:46.142278+0000","last_peered":"2024-09-11T08:00:46.142278+0000","last_clean":"2024-09-11T08:00:46.142278+0000","last_became_active":"2024-09-11T08:00:42.829363+0000","last_became_peered":"2024-09-11T08:00:42.829363+0000","last_unstale":"2024-09-11T08:00:46.142278+0000","last_undegraded":"2024-09-11T08:00:46.142278+0000","last_fullsized":"2024-09-11T08:00:46.142278+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:11:33.273548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144622+0000","last_change":"2024-09-11T08:00:38.811353+0000","last_active":"2024-09-11T08:00:46.144622+0000","last_peered":"2024-09-11T08:00:46.144622+0000","last_clean":"2024-09-11T08:00:46.144622+0000","last_became_active":"2024-09-11T08:00:38.811069+0000","last_became_peered":"2024-09-11T08:00:38.811069+0000","last_unstale":"2024-09-11T08:00:46.144622+0000","last_undegraded":"2024-09-11T08:00:46.144622+0000","last_fullsized":"2024-09-11T08:00:46.144622+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:33.934806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.200421+0000","last_change":"2024-09-11T08:00:36.803793+0000","last_active":"2024-09-11T08:00:46.200421+0000","last_peered":"2024-09-11T08:00:46.200421+0000","last_clean":"2024-09-11T08:00:46.200421+0000","last_became_active":"2024-09-11T08:00:36.803536+0000","last_became_peered":"2024-09-11T08:00:36.803536+0000","last_unstale":"2024-09-11T08:00:46.200421+0000","last_undegraded":"2024-09-11T08:00:46.200421+0000","last_fullsized":"2024-09-11T08:00:46.200421+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:20:44.229786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222245+0000","last_change":"2024-09-11T08:00:40.831179+0000","last_active":"2024-09-11T08:00:46.222245+0000","last_peered":"2024-09-11T08:00:46.222245+0000","last_clean":"2024-09-11T08:00:46.222245+0000","last_became_active":"2024-09-11T08:00:40.830817+0000","last_became_peered":"2024-09-11T08:00:40.830817+0000","last_unstale":"2024-09-11T08:00:46.222245+0000","last_undegraded":"2024-09-11T08:00:46.222245+0000","last_fullsized":"2024-09-11T08:00:46.222245+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:32:09.861168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143065+0000","last_change":"2024-09-11T08:00:42.835761+0000","last_active":"2024-09-11T08:00:46.143065+0000","last_peered":"2024-09-11T08:00:46.143065+0000","last_clean":"2024-09-11T08:00:46.143065+0000","last_became_active":"2024-09-11T08:00:42.835437+0000","last_became_peered":"2024-09-11T08:00:42.835437+0000","last_unstale":"2024-09-11T08:00:46.143065+0000","last_undegraded":"2024-09-11T08:00:46.143065+0000","last_fullsized":"2024-09-11T08:00:46.143065+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:04:24.902266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.141966+0000","last_change":"2024-09-11T08:00:38.807375+0000","last_active":"2024-09-11T08:00:46.141966+0000","last_peered":"2024-09-11T08:00:46.141966+0000","last_clean":"2024-09-11T08:00:46.141966+0000","last_became_active":"2024-09-11T08:00:38.806989+0000","last_became_peered":"2024-09-11T08:00:38.806989+0000","last_unstale":"2024-09-11T08:00:46.141966+0000","last_undegraded":"2024-09-11T08:00:46.141966+0000","last_fullsized":"2024-09-11T08:00:46.141966+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:03:48.987467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.145098+0000","last_change":"2024-09-11T08:00:36.808566+0000","last_active":"2024-09-11T08:00:46.145098+0000","last_peered":"2024-09-11T08:00:46.145098+0000","last_clean":"2024-09-11T08:00:46.145098+0000","last_became_active":"2024-09-11T08:00:36.808007+0000","last_became_peered":"2024-09-11T08:00:36.808007+0000","last_unstale":"2024-09-11T08:00:46.145098+0000","last_undegraded":"2024-09-11T08:00:46.145098+0000","last_fullsized":"2024-09-11T08:00:46.145098+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:41:47.056360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143551+0000","last_change":"2024-09-11T08:00:40.830750+0000","last_active":"2024-09-11T08:00:46.143551+0000","last_peered":"2024-09-11T08:00:46.143551+0000","last_clean":"2024-09-11T08:00:46.143551+0000","last_became_active":"2024-09-11T08:00:40.830425+0000","last_became_peered":"2024-09-11T08:00:40.830425+0000","last_unstale":"2024-09-11T08:00:46.143551+0000","last_undegraded":"2024-09-11T08:00:46.143551+0000","last_fullsized":"2024-09-11T08:00:46.143551+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:45:40.883179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.221867+0000","last_change":"2024-09-11T08:00:42.836560+0000","last_active":"2024-09-11T08:00:46.221867+0000","last_peered":"2024-09-11T08:00:46.221867+0000","last_clean":"2024-09-11T08:00:46.221867+0000","last_became_active":"2024-09-11T08:00:42.836301+0000","last_became_peered":"2024-09-11T08:00:42.836301+0000","last_unstale":"2024-09-11T08:00:46.221867+0000","last_undegraded":"2024-09-11T08:00:46.221867+0000","last_fullsized":"2024-09-11T08:00:46.221867+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:34.297728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"59'4","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.221846+0000","last_change":"2024-09-11T08:00:38.820225+0000","last_active":"2024-09-11T08:00:46.221846+0000","last_peered":"2024-09-11T08:00:46.221846+0000","last_clean":"2024-09-11T08:00:46.221846+0000","last_became_active":"2024-09-11T08:00:38.819965+0000","last_became_peered":"2024-09-11T08:00:38.819965+0000","last_unstale":"2024-09-11T08:00:46.221846+0000","last_undegraded":"2024-09-11T08:00:46.221846+0000","last_fullsized":"2024-09-11T08:00:46.221846+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:35:12.218080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227996+0000","last_change":"2024-09-11T08:00:36.804017+0000","last_active":"2024-09-11T08:00:46.227996+0000","last_peered":"2024-09-11T08:00:46.227996+0000","last_clean":"2024-09-11T08:00:46.227996+0000","last_became_active":"2024-09-11T08:00:36.803760+0000","last_became_peered":"2024-09-11T08:00:36.803760+0000","last_unstale":"2024-09-11T08:00:46.227996+0000","last_undegraded":"2024-09-11T08:00:46.227996+0000","last_fullsized":"2024-09-11T08:00:46.227996+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:45:42.732251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.200230+0000","last_change":"2024-09-11T08:00:40.834099+0000","last_active":"2024-09-11T08:00:46.200230+0000","last_peered":"2024-09-11T08:00:46.200230+0000","last_clean":"2024-09-11T08:00:46.200230+0000","last_became_active":"2024-09-11T08:00:40.833782+0000","last_became_peered":"2024-09-11T08:00:40.833782+0000","last_unstale":"2024-09-11T08:00:46.200230+0000","last_undegraded":"2024-09-11T08:00:46.200230+0000","last_fullsized":"2024-09-11T08:00:46.200230+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:02:44.475232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226613+0000","last_change":"2024-09-11T08:00:42.834547+0000","last_active":"2024-09-11T08:00:46.226613+0000","last_peered":"2024-09-11T08:00:46.226613+0000","last_clean":"2024-09-11T08:00:46.226613+0000","last_became_active":"2024-09-11T08:00:42.834148+0000","last_became_peered":"2024-09-11T08:00:42.834148+0000","last_unstale":"2024-09-11T08:00:46.226613+0000","last_undegraded":"2024-09-11T08:00:46.226613+0000","last_fullsized":"2024-09-11T08:00:46.226613+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:05:16.241381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222031+0000","last_change":"2024-09-11T08:00:39.123191+0000","last_active":"2024-09-11T08:00:46.222031+0000","last_peered":"2024-09-11T08:00:46.222031+0000","last_clean":"2024-09-11T08:00:46.222031+0000","last_became_active":"2024-09-11T08:00:39.122951+0000","last_became_peered":"2024-09-11T08:00:39.122951+0000","last_unstale":"2024-09-11T08:00:46.222031+0000","last_undegraded":"2024-09-11T08:00:46.222031+0000","last_fullsized":"2024-09-11T08:00:46.222031+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:37:03.917433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.200724+0000","last_change":"2024-09-11T08:00:36.801849+0000","last_active":"2024-09-11T08:00:46.200724+0000","last_peered":"2024-09-11T08:00:46.200724+0000","last_clean":"2024-09-11T08:00:46.200724+0000","last_became_active":"2024-09-11T08:00:36.800948+0000","last_became_peered":"2024-09-11T08:00:36.800948+0000","last_unstale":"2024-09-11T08:00:46.200724+0000","last_undegraded":"2024-09-11T08:00:46.200724+0000","last_fullsized":"2024-09-11T08:00:46.200724+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:53:32.454700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226525+0000","last_change":"2024-09-11T08:00:40.834066+0000","last_active":"2024-09-11T08:00:46.226525+0000","last_peered":"2024-09-11T08:00:46.226525+0000","last_clean":"2024-09-11T08:00:46.226525+0000","last_became_active":"2024-09-11T08:00:40.833675+0000","last_became_peered":"2024-09-11T08:00:40.833675+0000","last_unstale":"2024-09-11T08:00:46.226525+0000","last_undegraded":"2024-09-11T08:00:46.226525+0000","last_fullsized":"2024-09-11T08:00:46.226525+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:23:18.250635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222102+0000","last_change":"2024-09-11T08:00:43.068452+0000","last_active":"2024-09-11T08:00:46.222102+0000","last_peered":"2024-09-11T08:00:46.222102+0000","last_clean":"2024-09-11T08:00:46.222102+0000","last_became_active":"2024-09-11T08:00:43.068135+0000","last_became_peered":"2024-09-11T08:00:43.068135+0000","last_unstale":"2024-09-11T08:00:46.222102+0000","last_undegraded":"2024-09-11T08:00:46.222102+0000","last_fullsized":"2024-09-11T08:00:46.222102+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:23.938820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227466+0000","last_change":"2024-09-11T08:00:42.835011+0000","last_active":"2024-09-11T08:00:46.227466+0000","last_peered":"2024-09-11T08:00:46.227466+0000","last_clean":"2024-09-11T08:00:46.227466+0000","last_became_active":"2024-09-11T08:00:42.834758+0000","last_became_peered":"2024-09-11T08:00:42.834758+0000","last_unstale":"2024-09-11T08:00:46.227466+0000","last_undegraded":"2024-09-11T08:00:46.227466+0000","last_fullsized":"2024-09-11T08:00:46.227466+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:01:07.211070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226955+0000","last_change":"2024-09-11T08:00:36.801605+0000","last_active":"2024-09-11T08:00:46.226955+0000","last_peered":"2024-09-11T08:00:46.226955+0000","last_clean":"2024-09-11T08:00:46.226955+0000","last_became_active":"2024-09-11T08:00:36.801324+0000","last_became_peered":"2024-09-11T08:00:36.801324+0000","last_unstale":"2024-09-11T08:00:46.226955+0000","last_undegraded":"2024-09-11T08:00:46.226955+0000","last_fullsized":"2024-09-11T08:00:46.226955+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:53:36.775862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143732+0000","last_change":"2024-09-11T08:00:39.123777+0000","last_active":"2024-09-11T08:00:46.143732+0000","last_peered":"2024-09-11T08:00:46.143732+0000","last_clean":"2024-09-11T08:00:46.143732+0000","last_became_active":"2024-09-11T08:00:39.123477+0000","last_became_peered":"2024-09-11T08:00:39.123477+0000","last_unstale":"2024-09-11T08:00:46.143732+0000","last_undegraded":"2024-09-11T08:00:46.143732+0000","last_fullsized":"2024-09-11T08:00:46.143732+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:54:29.890493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"59'8","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:01:50.144375+0000","last_change":"2024-09-11T08:00:40.840322+0000","last_active":"2024-09-11T08:01:50.144375+0000","last_peered":"2024-09-11T08:01:50.144375+0000","last_clean":"2024-09-11T08:01:50.144375+0000","last_became_active":"2024-09-11T08:00:40.840008+0000","last_became_peered":"2024-09-11T08:00:40.840008+0000","last_unstale":"2024-09-11T08:01:50.144375+0000","last_undegraded":"2024-09-11T08:01:50.144375+0000","last_fullsized":"2024-09-11T08:01:50.144375+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:03.732737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143755+0000","last_change":"2024-09-11T08:00:42.831914+0000","last_active":"2024-09-11T08:00:46.143755+0000","last_peered":"2024-09-11T08:00:46.143755+0000","last_clean":"2024-09-11T08:00:46.143755+0000","last_became_active":"2024-09-11T08:00:42.831600+0000","last_became_peered":"2024-09-11T08:00:42.831600+0000","last_unstale":"2024-09-11T08:00:46.143755+0000","last_undegraded":"2024-09-11T08:00:46.143755+0000","last_fullsized":"2024-09-11T08:00:46.143755+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:14:02.421234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222508+0000","last_change":"2024-09-11T08:00:36.808458+0000","last_active":"2024-09-11T08:00:46.222508+0000","last_peered":"2024-09-11T08:00:46.222508+0000","last_clean":"2024-09-11T08:00:46.222508+0000","last_became_active":"2024-09-11T08:00:36.808188+0000","last_became_peered":"2024-09-11T08:00:36.808188+0000","last_unstale":"2024-09-11T08:00:46.222508+0000","last_undegraded":"2024-09-11T08:00:46.222508+0000","last_fullsized":"2024-09-11T08:00:46.222508+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:52:58.705799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.200654+0000","last_change":"2024-09-11T08:00:38.815295+0000","last_active":"2024-09-11T08:00:46.200654+0000","last_peered":"2024-09-11T08:00:46.200654+0000","last_clean":"2024-09-11T08:00:46.200654+0000","last_became_active":"2024-09-11T08:00:38.814976+0000","last_became_peered":"2024-09-11T08:00:38.814976+0000","last_unstale":"2024-09-11T08:00:46.200654+0000","last_undegraded":"2024-09-11T08:00:46.200654+0000","last_fullsized":"2024-09-11T08:00:46.200654+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:39:32.310184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143324+0000","last_change":"2024-09-11T08:00:40.834883+0000","last_active":"2024-09-11T08:00:46.143324+0000","last_peered":"2024-09-11T08:00:46.143324+0000","last_clean":"2024-09-11T08:00:46.143324+0000","last_became_active":"2024-09-11T08:00:40.834570+0000","last_became_peered":"2024-09-11T08:00:40.834570+0000","last_unstale":"2024-09-11T08:00:46.143324+0000","last_undegraded":"2024-09-11T08:00:46.143324+0000","last_fullsized":"2024-09-11T08:00:46.143324+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:27:26.508977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.142416+0000","last_change":"2024-09-11T08:00:43.067972+0000","last_active":"2024-09-11T08:00:46.142416+0000","last_peered":"2024-09-11T08:00:46.142416+0000","last_clean":"2024-09-11T08:00:46.142416+0000","last_became_active":"2024-09-11T08:00:43.067705+0000","last_became_peered":"2024-09-11T08:00:43.067705+0000","last_unstale":"2024-09-11T08:00:46.142416+0000","last_undegraded":"2024-09-11T08:00:46.142416+0000","last_fullsized":"2024-09-11T08:00:46.142416+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:03:23.242396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.200427+0000","last_change":"2024-09-11T08:00:36.798078+0000","last_active":"2024-09-11T08:00:46.200427+0000","last_peered":"2024-09-11T08:00:46.200427+0000","last_clean":"2024-09-11T08:00:46.200427+0000","last_became_active":"2024-09-11T08:00:36.797823+0000","last_became_peered":"2024-09-11T08:00:36.797823+0000","last_unstale":"2024-09-11T08:00:46.200427+0000","last_undegraded":"2024-09-11T08:00:46.200427+0000","last_fullsized":"2024-09-11T08:00:46.200427+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:04.709230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227208+0000","last_change":"2024-09-11T08:00:38.818066+0000","last_active":"2024-09-11T08:00:46.227208+0000","last_peered":"2024-09-11T08:00:46.227208+0000","last_clean":"2024-09-11T08:00:46.227208+0000","last_became_active":"2024-09-11T08:00:38.817773+0000","last_became_peered":"2024-09-11T08:00:38.817773+0000","last_unstale":"2024-09-11T08:00:46.227208+0000","last_undegraded":"2024-09-11T08:00:46.227208+0000","last_fullsized":"2024-09-11T08:00:46.227208+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:18:09.442656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144321+0000","last_change":"2024-09-11T08:00:40.835851+0000","last_active":"2024-09-11T08:00:46.144321+0000","last_peered":"2024-09-11T08:00:46.144321+0000","last_clean":"2024-09-11T08:00:46.144321+0000","last_became_active":"2024-09-11T08:00:40.835507+0000","last_became_peered":"2024-09-11T08:00:40.835507+0000","last_unstale":"2024-09-11T08:00:46.144321+0000","last_undegraded":"2024-09-11T08:00:46.144321+0000","last_fullsized":"2024-09-11T08:00:46.144321+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:28:25.572828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222745+0000","last_change":"2024-09-11T08:00:43.069092+0000","last_active":"2024-09-11T08:00:46.222745+0000","last_peered":"2024-09-11T08:00:46.222745+0000","last_clean":"2024-09-11T08:00:46.222745+0000","last_became_active":"2024-09-11T08:00:43.067854+0000","last_became_peered":"2024-09-11T08:00:43.067854+0000","last_unstale":"2024-09-11T08:00:46.222745+0000","last_undegraded":"2024-09-11T08:00:46.222745+0000","last_fullsized":"2024-09-11T08:00:46.222745+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:19.226214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222401+0000","last_change":"2024-09-11T08:00:36.805092+0000","last_active":"2024-09-11T08:00:46.222401+0000","last_peered":"2024-09-11T08:00:46.222401+0000","last_clean":"2024-09-11T08:00:46.222401+0000","last_became_active":"2024-09-11T08:00:36.804684+0000","last_became_peered":"2024-09-11T08:00:36.804684+0000","last_unstale":"2024-09-11T08:00:46.222401+0000","last_undegraded":"2024-09-11T08:00:46.222401+0000","last_fullsized":"2024-09-11T08:00:46.222401+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:28.554646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222474+0000","last_change":"2024-09-11T08:00:38.819139+0000","last_active":"2024-09-11T08:00:46.222474+0000","last_peered":"2024-09-11T08:00:46.222474+0000","last_clean":"2024-09-11T08:00:46.222474+0000","last_became_active":"2024-09-11T08:00:38.818876+0000","last_became_peered":"2024-09-11T08:00:38.818876+0000","last_unstale":"2024-09-11T08:00:46.222474+0000","last_undegraded":"2024-09-11T08:00:46.222474+0000","last_fullsized":"2024-09-11T08:00:46.222474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:42:19.097088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.142060+0000","last_change":"2024-09-11T08:00:40.827187+0000","last_active":"2024-09-11T08:00:46.142060+0000","last_peered":"2024-09-11T08:00:46.142060+0000","last_clean":"2024-09-11T08:00:46.142060+0000","last_became_active":"2024-09-11T08:00:40.826414+0000","last_became_peered":"2024-09-11T08:00:40.826414+0000","last_unstale":"2024-09-11T08:00:46.142060+0000","last_undegraded":"2024-09-11T08:00:46.142060+0000","last_fullsized":"2024-09-11T08:00:46.142060+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:34:01.256624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":66,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430228,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298536,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446616,"statfs":{"total":95995035648,"available":95689334784,"internally_reserved":0,"allocated":1212416,"data_stored":823517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.48299999999999998}]},{"osd":1,"last update":"Wed Sep 11 08:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.42199999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.51700000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.45100000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.38700000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.437}]},{"osd":6,"last update":"Wed Sep 11 08:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.377}]}]},{"osd":6,"up_from":42,"seq":180388626456,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298516,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446636,"statfs":{"total":95995035648,"available":95689355264,"internally_reserved":0,"allocated":1191936,"data_stored":822916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:01:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.437}]},{"osd":1,"last update":"Wed Sep 11 08:01:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.42699999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:01:00 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.41699999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:01:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.38500000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:01:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.32300000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:01:00 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.55800000000000005}]},{"osd":7,"last update":"Wed Sep 11 08:01:27 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.34499999999999997}]}]},{"osd":5,"up_from":36,"seq":154618822687,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297932,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447220,"statfs":{"total":95995035648,"available":95689953280,"internally_reserved":0,"allocated":589824,"data_stored":231366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:01:42 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52400000000000002,"15min":0.52700000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.66000000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.499,"15min":0.503},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66600000000000004,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.313}]},{"osd":1,"last update":"Wed Sep 11 08:01:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.36299999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66600000000000004,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.38400000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:01:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.50900000000000001,"15min":0.50800000000000001},"min":{"1min":0.35399999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.51600000000000001,"15min":0.51900000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.42699999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:01:42 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.49299999999999999,"15min":0.496},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.624,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51500000000000001,"15min":0.51300000000000001},"min":{"1min":0.38100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.47399999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:01:42 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38700000000000001,"15min":0.38800000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.58199999999999996,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.39800000000000002,"15min":0.39200000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.35399999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.40799999999999997}]},{"osd":7,"last update":"Wed Sep 11 08:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.437},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.46100000000000002}]}]},{"osd":4,"up_from":30,"seq":128849018915,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297956,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447196,"statfs":{"total":95995035648,"available":95689928704,"internally_reserved":0,"allocated":614400,"data_stored":232113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.499,"15min":0.497},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64700000000000002,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.47599999999999998,"15min":0.47299999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.47099999999999997}]},{"osd":1,"last update":"Wed Sep 11 08:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.76000000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.48099999999999998,"15min":0.47599999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.45900000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.51900000000000002,"15min":0.51600000000000001},"min":{"1min":0.36099999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.45300000000000001}]},{"osd":3,"last update":"Wed Sep 11 08:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.503,"15min":0.499},"min":{"1min":0.41799999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67200000000000004,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.42599999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.36299999999999999,"15min":0.35299999999999998},"min":{"1min":0.27800000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.39800000000000002,"15min":0.39000000000000001},"min":{"1min":0.312,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.41199999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:01:00 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.36599999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.434},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.33700000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182439,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297972,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447180,"statfs":{"total":95995035648,"available":95689912320,"internally_reserved":0,"allocated":630784,"data_stored":232464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36499999999999999,"15min":0.36399999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.214},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.26800000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.19900000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.41499999999999998,"15min":0.41599999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.371,"5min":0.39600000000000002,"15min":0.40000000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.49099999999999999,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.27300000000000002}]},{"osd":2,"last update":"Wed Sep 11 08:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.42299999999999999,"15min":0.42699999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.53400000000000003,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.441,"15min":0.44400000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.255}]},{"osd":4,"last update":"Wed Sep 11 08:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.35999999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73899999999999999,"5min":0.753,"15min":0.753},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.53700000000000003,"15min":0.54000000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71599999999999997,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.38900000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.314,"15min":0.314},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.377},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56100000000000005,"15min":0.56799999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73499999999999999,"5min":0.751,"15min":0.751},"last":0.36799999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:01:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.45500000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:01:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.443},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.40799999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411372,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297940,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447212,"statfs":{"total":95995035648,"available":95689945088,"internally_reserved":0,"allocated":598016,"data_stored":233095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.35499999999999998,"15min":0.36199999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.5,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.34899999999999998,"15min":0.34799999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.49399999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.26600000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.36299999999999999,"15min":0.36599999999999999},"min":{"1min":0.28999999999999998,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.443,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47099999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.34399999999999997}]},{"osd":3,"last update":"Wed Sep 11 08:00:56 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.64600000000000002,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.38900000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:01:16 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52500000000000002,"15min":0.52700000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.54000000000000004,"15min":0.53700000000000003},"min":{"1min":0.42999999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.375}]},{"osd":5,"last update":"Wed Sep 11 08:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53700000000000003,"15min":0.53500000000000003},"min":{"1min":0.39300000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.53400000000000003}]},{"osd":6,"last update":"Wed Sep 11 08:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.48299999999999998}]},{"osd":7,"last update":"Wed Sep 11 08:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.41699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297996,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447156,"statfs":{"total":95995035648,"available":95689887744,"internally_reserved":0,"allocated":655360,"data_stored":233649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:01:11 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.33600000000000002,"15min":0.32000000000000001},"min":{"1min":0.217,"5min":0.19,"15min":0.19},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32800000000000001},"min":{"1min":0.24399999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.46000000000000002,"5min":0.504,"15min":0.504},"last":0.34100000000000003}]},{"osd":2,"last update":"Wed Sep 11 08:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.38200000000000001,"15min":0.376},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.52000000000000002,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.374},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.30099999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.54600000000000004,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.433}]},{"osd":3,"last update":"Wed Sep 11 08:00:58 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.39000000000000001,"15min":0.39200000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55300000000000005,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.40799999999999997,"15min":0.40999999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.39900000000000002}]},{"osd":4,"last update":"Wed Sep 11 08:01:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56299999999999994,"15min":0.56599999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83899999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.51900000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.42399999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.78200000000000003,"5min":0.875,"15min":0.875},"last":0.55000000000000004}]},{"osd":6,"last update":"Wed Sep 11 08:01:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.499},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.56000000000000005}]},{"osd":7,"last update":"Wed Sep 11 08:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.53000000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738421,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298532,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446620,"statfs":{"total":95995035648,"available":95689338880,"internally_reserved":0,"allocated":1208320,"data_stored":822387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:01:11 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.31900000000000001,"15min":0.30599999999999999},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.32100000000000001},"min":{"1min":0.24099999999999999,"5min":0.192,"15min":0.192},"max":{"1min":0.52000000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.47799999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:01:32 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.35399999999999998,"15min":0.35799999999999998},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.40200000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:00:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42799999999999999,"15min":0.42999999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.63200000000000001,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.434},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.42999999999999999,"15min":0.436},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.64500000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.45100000000000001}]},{"osd":4,"last update":"Wed Sep 11 08:01:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.39100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73299999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56100000000000005,"15min":0.56299999999999994},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73499999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.54300000000000004}]},{"osd":5,"last update":"Wed Sep 11 08:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.373,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.61299999999999999}]},{"osd":6,"last update":"Wed Sep 11 08:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.53000000000000003}]},{"osd":7,"last update":"Wed Sep 11 08:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.58199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:01:58.960 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T08:01:59.170 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.92290766Z level=info msg="Executing migration" id="create team table" 2024-09-11T08:01:59.170 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.923604879Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-11T08:01:59.170 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.924382235Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-11T08:01:59.170 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.925197096Z level=info msg="Executing migration" id="create team member table" 2024-09-11T08:01:59.170 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.925980434Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-11T08:01:59.170 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.926978302Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-11T08:01:59.170 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.927684112Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-11T08:01:59.170 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.928513835Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-11T08:01:59.170 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.929347364Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-11T08:01:59.171 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.930266156Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-11T08:01:59.171 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.931099808Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-11T08:01:59.171 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.931971949Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-11T08:01:59.171 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.932933015Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-11T08:01:59.171 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.933671206Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-11T08:01:59.171 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.934495121Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-11T08:01:59.171 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.935354164Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-11T08:01:59.171 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.936127768Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-11T08:01:59.171 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.93712199Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-11T08:01:59.171 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.937960412Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-11T08:01:59.173 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.938605357Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-11T08:01:59.173 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.939414261Z level=info msg="Executing migration" id="create tag table" 2024-09-11T08:01:59.173 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.940418903Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-11T08:01:59.173 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.941137727Z level=info msg="Executing migration" id="create login attempt table" 2024-09-11T08:01:59.173 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.941529575Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-11T08:01:59.173 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.942498204Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-11T08:01:59.173 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.943204249Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-11T08:01:59.173 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.946004118Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-11T08:01:59.173 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.946413226Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-11T08:01:59.173 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.949338061Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-11T08:01:59.173 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.952247364Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-11T08:01:59.174 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.954958123Z level=info msg="Executing migration" id="create user auth table" 2024-09-11T08:01:59.174 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.955518608Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-11T08:01:59.174 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.95826899Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-11T08:01:59.174 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.96099667Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-11T08:01:59.174 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.963987802Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-11T08:01:59.174 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.964804636Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-11T08:01:59.174 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.967457643Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-11T08:01:59.174 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.970421351Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-11T08:01:59.174 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.97308108Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-11T08:01:59.174 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.973841646Z level=info msg="Executing migration" id="create server_lock table" 2024-09-11T08:01:59.174 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.976684983Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-11T08:01:59.174 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.979404809Z level=info msg="Executing migration" id="create user auth token table" 2024-09-11T08:01:59.174 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.982239372Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-11T08:01:59.175 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.984992223Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-11T08:01:59.175 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.987629975Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-11T08:01:59.175 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.990428137Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-11T08:01:59.175 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.993009483Z level=info msg="Executing migration" id="create cache_data table" 2024-09-11T08:01:59.175 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.993448876Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-11T08:01:59.175 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.996338094Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-11T08:01:59.175 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:58.998917304Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-11T08:01:59.175 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.00159313Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-11T08:01:59.175 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.002052763Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-11T08:01:59.175 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.004783328Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-11T08:01:59.175 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.005244476Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-11T08:01:59.175 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.007513885Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-11T08:01:59.175 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.010305777Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-11T08:01:59.175 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.010626191Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-11T08:01:59.176 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.013414331Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-11T08:01:59.176 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.013858467Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-11T08:01:59.176 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.016092884Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-11T08:01:59.176 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.016568992Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-11T08:01:59.176 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.018713178Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-11T08:01:59.176 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.019176304Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-11T08:01:59.176 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.020015519Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-11T08:01:59.176 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.020811469Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-11T08:01:59.176 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.021625918Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-11T08:01:59.176 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.022045562Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-11T08:01:59.176 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.02288924Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-11T08:01:59.176 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.023579639Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-11T08:01:59.177 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.023985571Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-11T08:01:59.177 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.02482738Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-11T08:01:59.177 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.025265516Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-11T08:01:59.177 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.026139894Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-11T08:01:59.177 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.026933313Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-11T08:01:59.177 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.027755261Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-11T08:01:59.177 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.031165564Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-11T08:01:59.177 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.03447753Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-11T08:01:59.177 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.035321768Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-11T08:01:59.177 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.036124531Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-11T08:01:59.177 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.037162074Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-11T08:01:59.178 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.037655182Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-11T08:01:59.178 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.040331762Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-11T08:01:59.178 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.043194466Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-11T08:01:59.178 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.043728303Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-11T08:01:59.178 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.04647364Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-11T08:01:59.178 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.047389712Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-11T08:01:59.178 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.049328138Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-11T08:01:59.178 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.052145625Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-11T08:01:59.178 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.052786669Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-11T08:01:59.178 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.05551351Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-11T08:01:59.178 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.056380962Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-11T08:01:59.178 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.058420702Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-11T08:01:59.178 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.061057236Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-11T08:01:59.179 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.062048312Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-11T08:01:59.179 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.06503678Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-11T08:01:59.179 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.067653119Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-11T08:01:59.179 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.070621477Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-11T08:01:59.179 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.071110398Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-11T08:01:59.179 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.073585325Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-11T08:01:59.179 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.073942625Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-11T08:01:59.179 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.076437257Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-11T08:01:59.179 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.077312665Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-11T08:01:59.179 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.079836718Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-11T08:01:59.179 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.082274164Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-11T08:01:59.179 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.084993596Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-11T08:01:59.179 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.08537968Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-11T08:01:59.180 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.088148652Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-11T08:01:59.180 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.090774645Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-11T08:01:59.180 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.09330692Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-11T08:01:59.180 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.094200938Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-11T08:01:59.180 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.0970075Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-11T08:01:59.180 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.099505009Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-11T08:01:59.180 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.102111455Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-11T08:01:59.180 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.102743548Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-11T08:01:59.180 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.105445172Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-11T08:01:59.180 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.106483624Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-11T08:01:59.180 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.107150613Z level=info msg="Executing migration" id="create alert_image table" 2024-09-11T08:01:59.180 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.107582297Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-11T08:01:59.180 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.108516202Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-11T08:01:59.181 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.109347311Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-11T08:01:59.181 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.109985915Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-11T08:01:59.181 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.110401689Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-11T08:01:59.181 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.110477709Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-11T08:01:59.181 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.111174529Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-11T08:01:59.181 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.112121055Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-11T08:01:59.181 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.112881374Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-11T08:01:59.181 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.1129951Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-11T08:01:59.181 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.113102583Z level=info msg="alerts found to migrate" alerts=0 2024-09-11T08:01:59.182 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.113508904Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-11T08:01:59.182 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.114355449Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-11T08:01:59.182 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.115365292Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-11T08:01:59.182 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.116055281Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-11T08:01:59.182 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.116594692Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-11T08:01:59.182 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.117380408Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-11T08:01:59.182 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.118214225Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-11T08:01:59.182 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.119136029Z level=info msg="Executing migration" id="create data_keys table" 2024-09-11T08:01:59.182 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.119723316Z level=info msg="Executing migration" id="create secrets table" 2024-09-11T08:01:59.182 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.120706007Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-11T08:01:59.182 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.125306026Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-11T08:01:59.182 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.126445609Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-11T08:01:59.182 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.127239175Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-11T08:01:59.183 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.131683367Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-11T08:01:59.183 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.1358639Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-11T08:01:59.183 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.136875243Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-11T08:01:59.183 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.137493007Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-11T08:01:59.183 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.138302757Z level=info msg="Executing migration" id="create permission table" 2024-09-11T08:01:59.183 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.138696759Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-11T08:01:59.183 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.139559787Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-11T08:01:59.183 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.140001284Z level=info msg="Executing migration" id="create role table" 2024-09-11T08:01:59.183 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.140948429Z level=info msg="Executing migration" id="add column display_name" 2024-09-11T08:01:59.184 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.142077824Z level=info msg="Executing migration" id="add column group_name" 2024-09-11T08:01:59.184 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.143002012Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-11T08:01:59.184 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.143452134Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-11T08:01:59.184 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.14441907Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-11T08:01:59.184 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.145198307Z level=info msg="Executing migration" id="create team role table" 2024-09-11T08:01:59.184 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.147972833Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-11T08:01:59.184 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.148432214Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-11T08:01:59.184 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.151277044Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-11T08:01:59.184 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.154059358Z level=info msg="Executing migration" id="create user role table" 2024-09-11T08:01:59.184 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.156690249Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-11T08:01:59.184 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.157235359Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-11T08:01:59.184 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.160121739Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-11T08:01:59.184 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.162639975Z level=info msg="Executing migration" id="create builtin role table" 2024-09-11T08:01:59.185 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.165465281Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-11T08:01:59.185 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.165929724Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-11T08:01:59.185 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.168785588Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-11T08:01:59.185 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.171584228Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-11T08:01:59.229 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:01:59.434 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:01:59.434 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.174518003Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-11T08:01:59.434 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.174976797Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-11T08:01:59.434 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.175417595Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-11T08:01:59.435 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.17584753Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-11T08:01:59.435 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.176244551Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-11T08:01:59.435 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.17668293Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-11T08:01:59.435 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.177688114Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-11T08:01:59.435 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.180424753Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-11T08:01:59.435 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.180892269Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-11T08:01:59.435 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.181172665Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-11T08:01:59.435 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.181600594Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-11T08:01:59.435 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.182039034Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-11T08:01:59.435 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.182433598Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-11T08:01:59.435 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.182754148Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-11T08:01:59.436 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.183083168Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-11T08:01:59.436 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.183489082Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-11T08:01:59.436 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.183920804Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-11T08:01:59.436 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.18487566Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-11T08:01:59.436 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.185193665Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-11T08:01:59.436 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.185627717Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-11T08:01:59.437 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.186052902Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-11T08:01:59.437 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.186484535Z level=info msg="Executing migration" id="add correlation config column" 2024-09-11T08:01:59.437 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.187480566Z level=info msg="Executing migration" id="create entity_events table" 2024-09-11T08:01:59.437 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.187907657Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-11T08:01:59.437 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.188310956Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-11T08:01:59.437 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.188394785Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-11T08:01:59.437 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.188670768Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T08:01:59.437 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.18874651Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T08:01:59.437 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.188989573Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-11T08:01:59.437 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.189401758Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-11T08:01:59.437 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.19186951Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-11T08:01:59.437 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.192439455Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T08:01:59.437 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.195306594Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-11T08:01:59.438 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.197664249Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-11T08:01:59.438 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.200454563Z level=info msg="Executing migration" id="Drop public config table" 2024-09-11T08:01:59.438 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.200864228Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-11T08:01:59.438 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.20336665Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-11T08:01:59.438 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.203813177Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-11T08:01:59.438 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.205942284Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-11T08:01:59.438 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.206417107Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-11T08:01:59.438 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.209465877Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-11T08:01:59.438 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.210481442Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-11T08:01:59.438 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.213128883Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-11T08:01:59.438 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.215806603Z level=info msg="Executing migration" id="add share column" 2024-09-11T08:01:59.438 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.218336493Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-11T08:01:59.438 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.21877829Z level=info msg="Executing migration" id="create file table" 2024-09-11T08:01:59.438 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.22139686Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-11T08:01:59.438 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.224060462Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-11T08:01:59.439 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.22458171Z level=info msg="Executing migration" id="create file_meta table" 2024-09-11T08:01:59.439 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.224978037Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-11T08:01:59.439 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.225832312Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-11T08:01:59.439 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.226629806Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-11T08:01:59.439 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.227452763Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-11T08:01:59.439 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.227795854Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-11T08:01:59.439 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.228588435Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-11T08:01:59.439 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.229651954Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-11T08:01:59.439 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.229982385Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-11T08:01:59.439 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.230821638Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-11T08:01:59.439 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.231235027Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-11T08:01:59.439 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.232071094Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-11T08:01:59.440 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.232920295Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-11T08:01:59.440 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.234005923Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-11T08:01:59.440 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.235066242Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-11T08:01:59.440 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.235915473Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-11T08:01:59.440 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.245628226Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-11T08:01:59.440 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.246432012Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-11T08:01:59.440 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.246914611Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-11T08:01:59.440 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.250363422Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-11T08:01:59.440 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.251214949Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-11T08:01:59.440 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.252149712Z level=info msg="Executing migration" id="create folder table" 2024-09-11T08:01:59.440 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.252633428Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-11T08:01:59.441 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.253428716Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-11T08:01:59.441 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.254289698Z level=info msg="Executing migration" id="Update folder title length" 2024-09-11T08:01:59.441 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.255173434Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-11T08:01:59.441 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=migrator t=2024-09-11T08:01:59.255670967Z level=info msg="migrations completed" performed=485 skipped=0 duration=669.717421ms 2024-09-11T08:01:59.441 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=sqlstore t=2024-09-11T08:01:59.25608733Z level=info msg="Created default organization" 2024-09-11T08:01:59.441 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=plugin.loader t=2024-09-11T08:01:59.313910535Z level=info msg="Plugin registered" pluginID=input 2024-09-11T08:01:59.441 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=plugin.loader t=2024-09-11T08:01:59.341838506Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-11T08:01:59.441 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=plugin.loader t=2024-09-11T08:01:59.34185887Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-11T08:01:59.441 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=secrets t=2024-09-11T08:01:59.341893091Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-11T08:01:59.441 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=query_data t=2024-09-11T08:01:59.343551597Z level=info msg="Query Service initialization" 2024-09-11T08:01:59.441 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=live.push_http t=2024-09-11T08:01:59.345818334Z level=info msg="Live Push Gateway initialization" 2024-09-11T08:01:59.496 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:01:59 smithi102 systemd[1]: Starting Ceph node-exporter.a for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:01:59.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:59 smithi102 ceph-mon[29849]: from='client.24638 v1:172.21.15.102:0/786223335' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:01:59.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:59 smithi102 ceph-mon[29849]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:59.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:59 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:59.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:59 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:59.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:59 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:59.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:59 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:59.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:59 smithi102 ceph-mon[29849]: Deploying daemon node-exporter.a on smithi102 2024-09-11T08:01:59.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:59 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:01:59.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:01:59 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:59.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:59 smithi102 ceph-mon[41371]: from='client.24638 v1:172.21.15.102:0/786223335' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:01:59.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:59 smithi102 ceph-mon[41371]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:59.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:59 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:59.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:59 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:59.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:59 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:59.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:59 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:59.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:59 smithi102 ceph-mon[41371]: Deploying daemon node-exporter.a on smithi102 2024-09-11T08:01:59.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:59 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:01:59.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:01:59 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:59.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-mon[33158]: from='client.24638 v1:172.21.15.102:0/786223335' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:01:59.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-mon[33158]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:01:59.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:59.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:59.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:59.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:01:59.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-mon[33158]: Deploying daemon node-exporter.a on smithi102 2024-09-11T08:01:59.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:01:59.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:01:59 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:00.329 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:00 smithi102 bash[77557]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-11T08:02:00.434 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:02:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=infra.usagestats.collector t=2024-09-11T08:02:00.076762725Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-11T08:02:00.434 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:02:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=provisioning.datasources t=2024-09-11T08:02:00.07708752Z level=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-09-11T08:02:00.434 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:02:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=provisioning.datasources t=2024-09-11T08:02:00.10953733Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-11T08:02:00.434 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:02:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=provisioning.datasources t=2024-09-11T08:02:00.141017877Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-11T08:02:00.435 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:02:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=provisioning.alerting t=2024-09-11T08:02:00.162646194Z level=info msg="starting to provision alerting" 2024-09-11T08:02:00.435 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:02:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=provisioning.alerting t=2024-09-11T08:02:00.162664996Z level=info msg="finished to provision alerting" 2024-09-11T08:02:00.435 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:02:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=grafanaStorageLogger t=2024-09-11T08:02:00.162776686Z level=info msg="storage starting" 2024-09-11T08:02:00.435 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:02:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=ngalert.state.manager t=2024-09-11T08:02:00.163020687Z level=info msg="Warming state cache for startup" 2024-09-11T08:02:00.435 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:02:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=http.server t=2024-09-11T08:02:00.164779636Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-11T08:02:00.435 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:02:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=sqlstore.transactions t=2024-09-11T08:02:00.173672653Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T08:02:00.435 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:02:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=ngalert.state.manager t=2024-09-11T08:02:00.173798735Z level=info msg="State cache has been initialized" states=0 duration=10.775462ms 2024-09-11T08:02:00.435 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:02:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=ngalert.multiorg.alertmanager t=2024-09-11T08:02:00.173858196Z level=info msg="starting MultiOrg Alertmanager" 2024-09-11T08:02:00.435 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:02:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=ticker t=2024-09-11T08:02:00.173876496Z level=info msg=starting first_tick=2024-09-11T08:02:10Z 2024-09-11T08:02:00.435 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:02:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=sqlstore.transactions t=2024-09-11T08:02:00.211310278Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T08:02:00.435 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:02:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=sqlstore.transactions t=2024-09-11T08:02:00.251483434Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T08:02:00.435 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:02:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=sqlstore.transactions t=2024-09-11T08:02:00.261978469Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-11T08:02:00.584 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:00 smithi102 bash[77557]: Getting image source signatures 2024-09-11T08:02:00.584 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:00 smithi102 bash[77557]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-11T08:02:00.585 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:00 smithi102 bash[77557]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-11T08:02:00.585 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:00 smithi102 bash[77557]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-11T08:02:00.857 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:00 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:02:00.857 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:00 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:02:00.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.855+0000 7efdd5d2c640 1 -- 172.21.15.102:0/2014657305 >> v1:172.21.15.102:6790/0 conn(0x7efdd0105350 legacy=0x7efdd0107730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:02:00.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.855+0000 7efdd5d2c640 1 -- 172.21.15.102:0/2014657305 shutdown_connections 2024-09-11T08:02:00.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.855+0000 7efdd5d2c640 1 -- 172.21.15.102:0/2014657305 >> 172.21.15.102:0/2014657305 conn(0x7efdd00f8150 msgr2=0x7efdd00fa560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:02:00.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.855+0000 7efdd5d2c640 1 -- 172.21.15.102:0/2014657305 shutdown_connections 2024-09-11T08:02:00.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.855+0000 7efdd5d2c640 1 -- 172.21.15.102:0/2014657305 wait complete. 2024-09-11T08:02:00.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.856+0000 7efdd5d2c640 1 Processor -- start 2024-09-11T08:02:00.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.856+0000 7efdd5d2c640 1 -- start start 2024-09-11T08:02:00.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.856+0000 7efdd5d2c640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7efdd011beb0 con 0x7efdd0108ee0 2024-09-11T08:02:00.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.856+0000 7efdd5d2c640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7efdd011d070 con 0x7efdd0102590 2024-09-11T08:02:00.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdd5d2c640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7efdd011e270 con 0x7efdd0105350 2024-09-11T08:02:00.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdc7fff640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7efdd0105350 0x7efdd00fe970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:51544/0 (socket says 172.21.15.102:51544) 2024-09-11T08:02:00.927 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdcf7fe640 1 --1- >> v1:172.21.15.146:6789/0 conn(0x7efdd0102590 0x7efdd00fe260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.146:6789/0 says I am v1:172.21.15.102:44258/0 (socket says 172.21.15.102:44258) 2024-09-11T08:02:00.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdc7fff640 1 -- 172.21.15.102:0/3705670856 learned_addr learned my addr 172.21.15.102:0/3705670856 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:02:00.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3662281298 0 0) 0x7efdd011d070 con 0x7efdd0102590 2024-09-11T08:02:00.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdbc0034e0 con 0x7efdd0102590 2024-09-11T08:02:00.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352258560 0 0) 0x7efdd011e270 con 0x7efdd0105350 2024-09-11T08:02:00.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdd011d070 con 0x7efdd0105350 2024-09-11T08:02:00.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3875782877 0 0) 0x7efdd011beb0 con 0x7efdd0108ee0 2024-09-11T08:02:00.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdd011e270 con 0x7efdd0108ee0 2024-09-11T08:02:00.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2060812710 0 0) 0x7efdbc0034e0 con 0x7efdd0102590 2024-09-11T08:02:00.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efdd011beb0 con 0x7efdd0102590 2024-09-11T08:02:00.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7efdc0001880 con 0x7efdd0102590 2024-09-11T08:02:00.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1884298715 0 0) 0x7efdd011d070 con 0x7efdd0105350 2024-09-11T08:02:00.928 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efdbc0034e0 con 0x7efdd0105350 2024-09-11T08:02:00.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7efdb80015d0 con 0x7efdd0105350 2024-09-11T08:02:00.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2671490538 0 0) 0x7efdd011e270 con 0x7efdd0108ee0 2024-09-11T08:02:00.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efdd011d070 con 0x7efdd0108ee0 2024-09-11T08:02:00.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 <== mon.1 v1:172.21.15.146:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3560254769 0 0) 0x7efdd011beb0 con 0x7efdd0102590 2024-09-11T08:02:00.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 >> v1:172.21.15.102:6790/0 conn(0x7efdd0105350 legacy=0x7efdd00fe970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:02:00.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 >> v1:172.21.15.102:6789/0 conn(0x7efdd0108ee0 legacy=0x7efdd011a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:02:00.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 --> v1:172.21.15.146:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd011f470 con 0x7efdd0102590 2024-09-11T08:02:00.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdd5d2c640 1 -- 172.21.15.102:0/3705670856 --> v1:172.21.15.146:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd011c0a0 con 0x7efdd0102590 2024-09-11T08:02:00.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.857+0000 7efdd5d2c640 1 -- 172.21.15.102:0/3705670856 --> v1:172.21.15.146:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd011c570 con 0x7efdd0102590 2024-09-11T08:02:00.929 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.858+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 <== mon.1 v1:172.21.15.146:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7efdc00021a0 con 0x7efdd0102590 2024-09-11T08:02:00.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.858+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 <== mon.1 v1:172.21.15.146:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7efdc0005700 con 0x7efdd0102590 2024-09-11T08:02:00.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.859+0000 7efdd5d2c640 1 -- 172.21.15.102:0/3705670856 --> v1:172.21.15.146:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb0005010 con 0x7efdd0102590 2024-09-11T08:02:00.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.859+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 <== mon.1 v1:172.21.15.146:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7efdc0001d50 con 0x7efdd0102590 2024-09-11T08:02:00.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.860+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 <== mon.1 v1:172.21.15.146:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 376332686 0 0) 0x7efdc0089ab0 con 0x7efdd0102590 2024-09-11T08:02:00.930 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:00.864+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 <== mon.1 v1:172.21.15.146:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7efdc00567a0 con 0x7efdd0102590 2024-09-11T08:02:00.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:00 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:02:01.036 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:01.035+0000 7efdd5d2c640 1 -- 172.21.15.102:0/3705670856 --> v1:172.21.15.102:6800/4281557748 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efdb0000b90 con 0x7efdbc06ff70 2024-09-11T08:02:01.046 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:01.045+0000 7efdccff9640 1 -- 172.21.15.102:0/3705670856 <== mgr.24497 v1:172.21.15.102:6800/4281557748 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+383730 (unknown 2965378022 0 29130974) 0x7efdb0000b90 con 0x7efdbc06ff70 2024-09-11T08:02:01.050 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T08:02:01.051 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-11T08:02:01.053 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:01.048+0000 7efdd5d2c640 1 -- 172.21.15.102:0/3705670856 >> v1:172.21.15.102:6800/4281557748 conn(0x7efdbc06ff70 legacy=0x7efdbc072420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:02:01.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:01.048+0000 7efdd5d2c640 1 -- 172.21.15.102:0/3705670856 >> v1:172.21.15.146:6789/0 conn(0x7efdd0102590 legacy=0x7efdd00fe260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:02:01.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:01.048+0000 7efdd5d2c640 1 -- 172.21.15.102:0/3705670856 shutdown_connections 2024-09-11T08:02:01.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:01.048+0000 7efdd5d2c640 1 -- 172.21.15.102:0/3705670856 >> 172.21.15.102:0/3705670856 conn(0x7efdd00f8150 msgr2=0x7efdd010c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:02:01.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:01.048+0000 7efdd5d2c640 1 -- 172.21.15.102:0/3705670856 shutdown_connections 2024-09-11T08:02:01.054 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:01.049+0000 7efdd5d2c640 1 -- 172.21.15.102:0/3705670856 wait complete. 2024-09-11T08:02:01.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:01 smithi102 ceph-mon[41371]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:01.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:01 smithi102 ceph-mon[29849]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:01.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:01 smithi146 ceph-mon[33158]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:02.381 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:02 smithi102 bash[77557]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-11T08:02:02.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:02 smithi102 ceph-mon[29849]: from='client.24655 v1:172.21.15.102:0/3705670856' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:02:02.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:02 smithi102 ceph-mon[41371]: from='client.24655 v1:172.21.15.102:0/3705670856' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:02:02.646 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:02 smithi102 bash[77557]: Writing manifest to image destination 2024-09-11T08:02:02.821 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-11T08:02:00.339153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":929,"num_read_kb":786,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385380,"kb_used_data":6544,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575836,"statfs":{"total":767960285184,"available":765517656064,"internally_reserved":0,"allocated":6701056,"data_stored":3631507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002580"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222044+0000","last_change":"2024-09-11T08:00:43.071118+0000","last_active":"2024-09-11T08:00:46.222044+0000","last_peered":"2024-09-11T08:00:46.222044+0000","last_clean":"2024-09-11T08:00:46.222044+0000","last_became_active":"2024-09-11T08:00:43.070866+0000","last_became_peered":"2024-09-11T08:00:43.070866+0000","last_unstale":"2024-09-11T08:00:46.222044+0000","last_undegraded":"2024-09-11T08:00:46.222044+0000","last_fullsized":"2024-09-11T08:00:46.222044+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:04.189160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.145008+0000","last_change":"2024-09-11T08:00:36.806461+0000","last_active":"2024-09-11T08:00:46.145008+0000","last_peered":"2024-09-11T08:00:46.145008+0000","last_clean":"2024-09-11T08:00:46.145008+0000","last_became_active":"2024-09-11T08:00:36.806166+0000","last_became_peered":"2024-09-11T08:00:36.806166+0000","last_unstale":"2024-09-11T08:00:46.145008+0000","last_undegraded":"2024-09-11T08:00:46.145008+0000","last_fullsized":"2024-09-11T08:00:46.145008+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:08.181846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222351+0000","last_change":"2024-09-11T08:00:38.821552+0000","last_active":"2024-09-11T08:00:46.222351+0000","last_peered":"2024-09-11T08:00:46.222351+0000","last_clean":"2024-09-11T08:00:46.222351+0000","last_became_active":"2024-09-11T08:00:38.821284+0000","last_became_peered":"2024-09-11T08:00:38.821284+0000","last_unstale":"2024-09-11T08:00:46.222351+0000","last_undegraded":"2024-09-11T08:00:46.222351+0000","last_fullsized":"2024-09-11T08:00:46.222351+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:29:25.936260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.141575+0000","last_change":"2024-09-11T08:00:40.833382+0000","last_active":"2024-09-11T08:00:46.141575+0000","last_peered":"2024-09-11T08:00:46.141575+0000","last_clean":"2024-09-11T08:00:46.141575+0000","last_became_active":"2024-09-11T08:00:40.833096+0000","last_became_peered":"2024-09-11T08:00:40.833096+0000","last_unstale":"2024-09-11T08:00:46.141575+0000","last_undegraded":"2024-09-11T08:00:46.141575+0000","last_fullsized":"2024-09-11T08:00:46.141575+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:33:53.440815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222758+0000","last_change":"2024-09-11T08:00:36.807450+0000","last_active":"2024-09-11T08:00:46.222758+0000","last_peered":"2024-09-11T08:00:46.222758+0000","last_clean":"2024-09-11T08:00:46.222758+0000","last_became_active":"2024-09-11T08:00:36.807201+0000","last_became_peered":"2024-09-11T08:00:36.807201+0000","last_unstale":"2024-09-11T08:00:46.222758+0000","last_undegraded":"2024-09-11T08:00:46.222758+0000","last_fullsized":"2024-09-11T08:00:46.222758+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:34:19.120603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.141584+0000","last_change":"2024-09-11T08:00:38.807270+0000","last_active":"2024-09-11T08:00:46.141584+0000","last_peered":"2024-09-11T08:00:46.141584+0000","last_clean":"2024-09-11T08:00:46.141584+0000","last_became_active":"2024-09-11T08:00:38.806941+0000","last_became_peered":"2024-09-11T08:00:38.806941+0000","last_unstale":"2024-09-11T08:00:46.141584+0000","last_undegraded":"2024-09-11T08:00:46.141584+0000","last_fullsized":"2024-09-11T08:00:46.141584+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:18:26.615405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143144+0000","last_change":"2024-09-11T08:00:40.835014+0000","last_active":"2024-09-11T08:00:46.143144+0000","last_peered":"2024-09-11T08:00:46.143144+0000","last_clean":"2024-09-11T08:00:46.143144+0000","last_became_active":"2024-09-11T08:00:40.834514+0000","last_became_peered":"2024-09-11T08:00:40.834514+0000","last_unstale":"2024-09-11T08:00:46.143144+0000","last_undegraded":"2024-09-11T08:00:46.143144+0000","last_fullsized":"2024-09-11T08:00:46.143144+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:41:53.217112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.141805+0000","last_change":"2024-09-11T08:00:42.830798+0000","last_active":"2024-09-11T08:00:46.141805+0000","last_peered":"2024-09-11T08:00:46.141805+0000","last_clean":"2024-09-11T08:00:46.141805+0000","last_became_active":"2024-09-11T08:00:42.830511+0000","last_became_peered":"2024-09-11T08:00:42.830511+0000","last_unstale":"2024-09-11T08:00:46.141805+0000","last_undegraded":"2024-09-11T08:00:46.141805+0000","last_fullsized":"2024-09-11T08:00:46.141805+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:47:47.676397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227650+0000","last_change":"2024-09-11T08:00:36.809729+0000","last_active":"2024-09-11T08:00:46.227650+0000","last_peered":"2024-09-11T08:00:46.227650+0000","last_clean":"2024-09-11T08:00:46.227650+0000","last_became_active":"2024-09-11T08:00:36.809471+0000","last_became_peered":"2024-09-11T08:00:36.809471+0000","last_unstale":"2024-09-11T08:00:46.227650+0000","last_undegraded":"2024-09-11T08:00:46.227650+0000","last_fullsized":"2024-09-11T08:00:46.227650+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:58.420627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"59'5","reported_seq":26,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.141896+0000","last_change":"2024-09-11T08:00:38.814013+0000","last_active":"2024-09-11T08:00:46.141896+0000","last_peered":"2024-09-11T08:00:46.141896+0000","last_clean":"2024-09-11T08:00:46.141896+0000","last_became_active":"2024-09-11T08:00:38.813580+0000","last_became_peered":"2024-09-11T08:00:38.813580+0000","last_unstale":"2024-09-11T08:00:46.141896+0000","last_undegraded":"2024-09-11T08:00:46.141896+0000","last_fullsized":"2024-09-11T08:00:46.141896+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:37:34.146632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227002+0000","last_change":"2024-09-11T08:00:40.835066+0000","last_active":"2024-09-11T08:00:46.227002+0000","last_peered":"2024-09-11T08:00:46.227002+0000","last_clean":"2024-09-11T08:00:46.227002+0000","last_became_active":"2024-09-11T08:00:40.833127+0000","last_became_peered":"2024-09-11T08:00:40.833127+0000","last_unstale":"2024-09-11T08:00:46.227002+0000","last_undegraded":"2024-09-11T08:00:46.227002+0000","last_fullsized":"2024-09-11T08:00:46.227002+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:02:23.340171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226723+0000","last_change":"2024-09-11T08:00:42.831044+0000","last_active":"2024-09-11T08:00:46.226723+0000","last_peered":"2024-09-11T08:00:46.226723+0000","last_clean":"2024-09-11T08:00:46.226723+0000","last_became_active":"2024-09-11T08:00:42.830649+0000","last_became_peered":"2024-09-11T08:00:42.830649+0000","last_unstale":"2024-09-11T08:00:46.226723+0000","last_undegraded":"2024-09-11T08:00:46.226723+0000","last_fullsized":"2024-09-11T08:00:46.226723+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:15:35.987319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227737+0000","last_change":"2024-09-11T08:00:36.802253+0000","last_active":"2024-09-11T08:00:46.227737+0000","last_peered":"2024-09-11T08:00:46.227737+0000","last_clean":"2024-09-11T08:00:46.227737+0000","last_became_active":"2024-09-11T08:00:36.801488+0000","last_became_peered":"2024-09-11T08:00:36.801488+0000","last_unstale":"2024-09-11T08:00:46.227737+0000","last_undegraded":"2024-09-11T08:00:46.227737+0000","last_fullsized":"2024-09-11T08:00:46.227737+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:36:37.539262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.141921+0000","last_change":"2024-09-11T08:00:38.817584+0000","last_active":"2024-09-11T08:00:46.141921+0000","last_peered":"2024-09-11T08:00:46.141921+0000","last_clean":"2024-09-11T08:00:46.141921+0000","last_became_active":"2024-09-11T08:00:38.817293+0000","last_became_peered":"2024-09-11T08:00:38.817293+0000","last_unstale":"2024-09-11T08:00:46.141921+0000","last_undegraded":"2024-09-11T08:00:46.141921+0000","last_fullsized":"2024-09-11T08:00:46.141921+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:07:27.412741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226809+0000","last_change":"2024-09-11T08:00:40.840328+0000","last_active":"2024-09-11T08:00:46.226809+0000","last_peered":"2024-09-11T08:00:46.226809+0000","last_clean":"2024-09-11T08:00:46.226809+0000","last_became_active":"2024-09-11T08:00:40.840015+0000","last_became_peered":"2024-09-11T08:00:40.840015+0000","last_unstale":"2024-09-11T08:00:46.226809+0000","last_undegraded":"2024-09-11T08:00:46.226809+0000","last_fullsized":"2024-09-11T08:00:46.226809+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:13:51.451498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143377+0000","last_change":"2024-09-11T08:00:42.834881+0000","last_active":"2024-09-11T08:00:46.143377+0000","last_peered":"2024-09-11T08:00:46.143377+0000","last_clean":"2024-09-11T08:00:46.143377+0000","last_became_active":"2024-09-11T08:00:42.834526+0000","last_became_peered":"2024-09-11T08:00:42.834526+0000","last_unstale":"2024-09-11T08:00:46.143377+0000","last_undegraded":"2024-09-11T08:00:46.143377+0000","last_fullsized":"2024-09-11T08:00:46.143377+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:56:20.510162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"59'17","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.142268+0000","last_change":"2024-09-11T08:00:38.810541+0000","last_active":"2024-09-11T08:00:46.142268+0000","last_peered":"2024-09-11T08:00:46.142268+0000","last_clean":"2024-09-11T08:00:46.142268+0000","last_became_active":"2024-09-11T08:00:38.810233+0000","last_became_peered":"2024-09-11T08:00:38.810233+0000","last_unstale":"2024-09-11T08:00:46.142268+0000","last_undegraded":"2024-09-11T08:00:46.142268+0000","last_fullsized":"2024-09-11T08:00:46.142268+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:01:30.198295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227468+0000","last_change":"2024-09-11T08:00:36.802119+0000","last_active":"2024-09-11T08:00:46.227468+0000","last_peered":"2024-09-11T08:00:46.227468+0000","last_clean":"2024-09-11T08:00:46.227468+0000","last_became_active":"2024-09-11T08:00:36.801499+0000","last_became_peered":"2024-09-11T08:00:36.801499+0000","last_unstale":"2024-09-11T08:00:46.227468+0000","last_undegraded":"2024-09-11T08:00:46.227468+0000","last_fullsized":"2024-09-11T08:00:46.227468+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:19:04.266476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.145179+0000","last_change":"2024-09-11T08:00:40.835188+0000","last_active":"2024-09-11T08:00:46.145179+0000","last_peered":"2024-09-11T08:00:46.145179+0000","last_clean":"2024-09-11T08:00:46.145179+0000","last_became_active":"2024-09-11T08:00:40.834922+0000","last_became_peered":"2024-09-11T08:00:40.834922+0000","last_unstale":"2024-09-11T08:00:46.145179+0000","last_undegraded":"2024-09-11T08:00:46.145179+0000","last_fullsized":"2024-09-11T08:00:46.145179+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:07:44.376883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227296+0000","last_change":"2024-09-11T08:00:42.835330+0000","last_active":"2024-09-11T08:00:46.227296+0000","last_peered":"2024-09-11T08:00:46.227296+0000","last_clean":"2024-09-11T08:00:46.227296+0000","last_became_active":"2024-09-11T08:00:42.835044+0000","last_became_peered":"2024-09-11T08:00:42.835044+0000","last_unstale":"2024-09-11T08:00:46.227296+0000","last_undegraded":"2024-09-11T08:00:46.227296+0000","last_fullsized":"2024-09-11T08:00:46.227296+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:56:19.084364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.142085+0000","last_change":"2024-09-11T08:00:38.813670+0000","last_active":"2024-09-11T08:00:46.142085+0000","last_peered":"2024-09-11T08:00:46.142085+0000","last_clean":"2024-09-11T08:00:46.142085+0000","last_became_active":"2024-09-11T08:00:38.813399+0000","last_became_peered":"2024-09-11T08:00:38.813399+0000","last_unstale":"2024-09-11T08:00:46.142085+0000","last_undegraded":"2024-09-11T08:00:46.142085+0000","last_fullsized":"2024-09-11T08:00:46.142085+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:28:56.757990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144882+0000","last_change":"2024-09-11T08:00:36.799756+0000","last_active":"2024-09-11T08:00:46.144882+0000","last_peered":"2024-09-11T08:00:46.144882+0000","last_clean":"2024-09-11T08:00:46.144882+0000","last_became_active":"2024-09-11T08:00:36.799313+0000","last_became_peered":"2024-09-11T08:00:36.799313+0000","last_unstale":"2024-09-11T08:00:46.144882+0000","last_undegraded":"2024-09-11T08:00:46.144882+0000","last_fullsized":"2024-09-11T08:00:46.144882+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:40:43.725089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:01:55.144550+0000","last_change":"2024-09-11T08:00:40.834916+0000","last_active":"2024-09-11T08:01:55.144550+0000","last_peered":"2024-09-11T08:01:55.144550+0000","last_clean":"2024-09-11T08:01:55.144550+0000","last_became_active":"2024-09-11T08:00:40.834572+0000","last_became_peered":"2024-09-11T08:00:40.834572+0000","last_unstale":"2024-09-11T08:01:55.144550+0000","last_undegraded":"2024-09-11T08:01:55.144550+0000","last_fullsized":"2024-09-11T08:01:55.144550+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:07:51.608546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.142422+0000","last_change":"2024-09-11T08:00:42.831080+0000","last_active":"2024-09-11T08:00:46.142422+0000","last_peered":"2024-09-11T08:00:46.142422+0000","last_clean":"2024-09-11T08:00:46.142422+0000","last_became_active":"2024-09-11T08:00:42.830716+0000","last_became_peered":"2024-09-11T08:00:42.830716+0000","last_unstale":"2024-09-11T08:00:46.142422+0000","last_undegraded":"2024-09-11T08:00:46.142422+0000","last_fullsized":"2024-09-11T08:00:46.142422+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:32:53.539884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144421+0000","last_change":"2024-09-11T08:00:38.816566+0000","last_active":"2024-09-11T08:00:46.144421+0000","last_peered":"2024-09-11T08:00:46.144421+0000","last_clean":"2024-09-11T08:00:46.144421+0000","last_became_active":"2024-09-11T08:00:38.816286+0000","last_became_peered":"2024-09-11T08:00:38.816286+0000","last_unstale":"2024-09-11T08:00:46.144421+0000","last_undegraded":"2024-09-11T08:00:46.144421+0000","last_fullsized":"2024-09-11T08:00:46.144421+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:07:00.156358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143336+0000","last_change":"2024-09-11T08:00:36.799352+0000","last_active":"2024-09-11T08:00:46.143336+0000","last_peered":"2024-09-11T08:00:46.143336+0000","last_clean":"2024-09-11T08:00:46.143336+0000","last_became_active":"2024-09-11T08:00:36.798975+0000","last_became_peered":"2024-09-11T08:00:36.798975+0000","last_unstale":"2024-09-11T08:00:46.143336+0000","last_undegraded":"2024-09-11T08:00:46.143336+0000","last_fullsized":"2024-09-11T08:00:46.143336+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:47:23.251644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:01:55.144666+0000","last_change":"2024-09-11T08:00:40.829051+0000","last_active":"2024-09-11T08:01:55.144666+0000","last_peered":"2024-09-11T08:01:55.144666+0000","last_clean":"2024-09-11T08:01:55.144666+0000","last_became_active":"2024-09-11T08:00:40.828790+0000","last_became_peered":"2024-09-11T08:00:40.828790+0000","last_unstale":"2024-09-11T08:01:55.144666+0000","last_undegraded":"2024-09-11T08:01:55.144666+0000","last_fullsized":"2024-09-11T08:01:55.144666+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:44:00.386495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.228188+0000","last_change":"2024-09-11T08:00:42.832968+0000","last_active":"2024-09-11T08:00:46.228188+0000","last_peered":"2024-09-11T08:00:46.228188+0000","last_clean":"2024-09-11T08:00:46.228188+0000","last_became_active":"2024-09-11T08:00:42.832644+0000","last_became_peered":"2024-09-11T08:00:42.832644+0000","last_unstale":"2024-09-11T08:00:46.228188+0000","last_undegraded":"2024-09-11T08:00:46.228188+0000","last_fullsized":"2024-09-11T08:00:46.228188+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:49.382900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.142506+0000","last_change":"2024-09-11T08:00:38.807986+0000","last_active":"2024-09-11T08:00:46.142506+0000","last_peered":"2024-09-11T08:00:46.142506+0000","last_clean":"2024-09-11T08:00:46.142506+0000","last_became_active":"2024-09-11T08:00:38.807500+0000","last_became_peered":"2024-09-11T08:00:38.807500+0000","last_unstale":"2024-09-11T08:00:46.142506+0000","last_undegraded":"2024-09-11T08:00:46.142506+0000","last_fullsized":"2024-09-11T08:00:46.142506+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:37:38.524178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226722+0000","last_change":"2024-09-11T08:00:36.809196+0000","last_active":"2024-09-11T08:00:46.226722+0000","last_peered":"2024-09-11T08:00:46.226722+0000","last_clean":"2024-09-11T08:00:46.226722+0000","last_became_active":"2024-09-11T08:00:36.808925+0000","last_became_peered":"2024-09-11T08:00:36.808925+0000","last_unstale":"2024-09-11T08:00:46.226722+0000","last_undegraded":"2024-09-11T08:00:46.226722+0000","last_fullsized":"2024-09-11T08:00:46.226722+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:40:06.627847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.228016+0000","last_change":"2024-09-11T08:00:40.832397+0000","last_active":"2024-09-11T08:00:46.228016+0000","last_peered":"2024-09-11T08:00:46.228016+0000","last_clean":"2024-09-11T08:00:46.228016+0000","last_became_active":"2024-09-11T08:00:40.832005+0000","last_became_peered":"2024-09-11T08:00:40.832005+0000","last_unstale":"2024-09-11T08:00:46.228016+0000","last_undegraded":"2024-09-11T08:00:46.228016+0000","last_fullsized":"2024-09-11T08:00:46.228016+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:44:41.009589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222270+0000","last_change":"2024-09-11T08:00:43.068306+0000","last_active":"2024-09-11T08:00:46.222270+0000","last_peered":"2024-09-11T08:00:46.222270+0000","last_clean":"2024-09-11T08:00:46.222270+0000","last_became_active":"2024-09-11T08:00:43.067787+0000","last_became_peered":"2024-09-11T08:00:43.067787+0000","last_unstale":"2024-09-11T08:00:46.222270+0000","last_undegraded":"2024-09-11T08:00:46.222270+0000","last_fullsized":"2024-09-11T08:00:46.222270+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:52.034174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"59'14","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.141657+0000","last_change":"2024-09-11T08:00:38.808225+0000","last_active":"2024-09-11T08:00:46.141657+0000","last_peered":"2024-09-11T08:00:46.141657+0000","last_clean":"2024-09-11T08:00:46.141657+0000","last_became_active":"2024-09-11T08:00:38.807792+0000","last_became_peered":"2024-09-11T08:00:38.807792+0000","last_unstale":"2024-09-11T08:00:46.141657+0000","last_undegraded":"2024-09-11T08:00:46.141657+0000","last_fullsized":"2024-09-11T08:00:46.141657+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:46:04.937830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"52'1","reported_seq":27,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.200735+0000","last_change":"2024-09-11T08:00:36.799406+0000","last_active":"2024-09-11T08:00:46.200735+0000","last_peered":"2024-09-11T08:00:46.200735+0000","last_clean":"2024-09-11T08:00:46.200735+0000","last_became_active":"2024-09-11T08:00:36.799038+0000","last_became_peered":"2024-09-11T08:00:36.799038+0000","last_unstale":"2024-09-11T08:00:46.200735+0000","last_undegraded":"2024-09-11T08:00:46.200735+0000","last_fullsized":"2024-09-11T08:00:46.200735+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:11:05.226318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:01:55.144537+0000","last_change":"2024-09-11T08:00:40.834172+0000","last_active":"2024-09-11T08:01:55.144537+0000","last_peered":"2024-09-11T08:01:55.144537+0000","last_clean":"2024-09-11T08:01:55.144537+0000","last_became_active":"2024-09-11T08:00:40.833864+0000","last_became_peered":"2024-09-11T08:00:40.833864+0000","last_unstale":"2024-09-11T08:01:55.144537+0000","last_undegraded":"2024-09-11T08:01:55.144537+0000","last_fullsized":"2024-09-11T08:01:55.144537+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:46:10.088736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227285+0000","last_change":"2024-09-11T08:00:43.070101+0000","last_active":"2024-09-11T08:00:46.227285+0000","last_peered":"2024-09-11T08:00:46.227285+0000","last_clean":"2024-09-11T08:00:46.227285+0000","last_became_active":"2024-09-11T08:00:43.069799+0000","last_became_peered":"2024-09-11T08:00:43.069799+0000","last_unstale":"2024-09-11T08:00:46.227285+0000","last_undegraded":"2024-09-11T08:00:46.227285+0000","last_fullsized":"2024-09-11T08:00:46.227285+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:42:56.848971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"59'18","reported_seq":48,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.221973+0000","last_change":"2024-09-11T08:00:39.123505+0000","last_active":"2024-09-11T08:00:46.221973+0000","last_peered":"2024-09-11T08:00:46.221973+0000","last_clean":"2024-09-11T08:00:46.221973+0000","last_became_active":"2024-09-11T08:00:39.123126+0000","last_became_peered":"2024-09-11T08:00:39.123126+0000","last_unstale":"2024-09-11T08:00:46.221973+0000","last_undegraded":"2024-09-11T08:00:46.221973+0000","last_fullsized":"2024-09-11T08:00:46.221973+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:13:50.011710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144754+0000","last_change":"2024-09-11T08:00:36.796896+0000","last_active":"2024-09-11T08:00:46.144754+0000","last_peered":"2024-09-11T08:00:46.144754+0000","last_clean":"2024-09-11T08:00:46.144754+0000","last_became_active":"2024-09-11T08:00:36.796626+0000","last_became_peered":"2024-09-11T08:00:36.796626+0000","last_unstale":"2024-09-11T08:00:46.144754+0000","last_undegraded":"2024-09-11T08:00:46.144754+0000","last_fullsized":"2024-09-11T08:00:46.144754+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:57:03.740594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.141575+0000","last_change":"2024-09-11T08:00:40.831495+0000","last_active":"2024-09-11T08:00:46.141575+0000","last_peered":"2024-09-11T08:00:46.141575+0000","last_clean":"2024-09-11T08:00:46.141575+0000","last_became_active":"2024-09-11T08:00:40.831201+0000","last_became_peered":"2024-09-11T08:00:40.831201+0000","last_unstale":"2024-09-11T08:00:46.141575+0000","last_undegraded":"2024-09-11T08:00:46.141575+0000","last_fullsized":"2024-09-11T08:00:46.141575+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:40.593930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.141954+0000","last_change":"2024-09-11T08:00:42.834169+0000","last_active":"2024-09-11T08:00:46.141954+0000","last_peered":"2024-09-11T08:00:46.141954+0000","last_clean":"2024-09-11T08:00:46.141954+0000","last_became_active":"2024-09-11T08:00:42.833883+0000","last_became_peered":"2024-09-11T08:00:42.833883+0000","last_unstale":"2024-09-11T08:00:46.141954+0000","last_undegraded":"2024-09-11T08:00:46.141954+0000","last_fullsized":"2024-09-11T08:00:46.141954+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:22:37.209284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"59'19","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144705+0000","last_change":"2024-09-11T08:00:39.123210+0000","last_active":"2024-09-11T08:00:46.144705+0000","last_peered":"2024-09-11T08:00:46.144705+0000","last_clean":"2024-09-11T08:00:46.144705+0000","last_became_active":"2024-09-11T08:00:39.122933+0000","last_became_peered":"2024-09-11T08:00:39.122933+0000","last_unstale":"2024-09-11T08:00:46.144705+0000","last_undegraded":"2024-09-11T08:00:46.144705+0000","last_fullsized":"2024-09-11T08:00:46.144705+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:09:07.029660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227720+0000","last_change":"2024-09-11T08:00:36.809169+0000","last_active":"2024-09-11T08:00:46.227720+0000","last_peered":"2024-09-11T08:00:46.227720+0000","last_clean":"2024-09-11T08:00:46.227720+0000","last_became_active":"2024-09-11T08:00:36.808895+0000","last_became_peered":"2024-09-11T08:00:36.808895+0000","last_unstale":"2024-09-11T08:00:46.227720+0000","last_undegraded":"2024-09-11T08:00:46.227720+0000","last_fullsized":"2024-09-11T08:00:46.227720+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:39:10.932536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.200272+0000","last_change":"2024-09-11T08:00:40.839308+0000","last_active":"2024-09-11T08:00:46.200272+0000","last_peered":"2024-09-11T08:00:46.200272+0000","last_clean":"2024-09-11T08:00:46.200272+0000","last_became_active":"2024-09-11T08:00:40.839007+0000","last_became_peered":"2024-09-11T08:00:40.839007+0000","last_unstale":"2024-09-11T08:00:46.200272+0000","last_undegraded":"2024-09-11T08:00:46.200272+0000","last_fullsized":"2024-09-11T08:00:46.200272+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:49:28.124681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143065+0000","last_change":"2024-09-11T08:00:43.068167+0000","last_active":"2024-09-11T08:00:46.143065+0000","last_peered":"2024-09-11T08:00:46.143065+0000","last_clean":"2024-09-11T08:00:46.143065+0000","last_became_active":"2024-09-11T08:00:43.067709+0000","last_became_peered":"2024-09-11T08:00:43.067709+0000","last_unstale":"2024-09-11T08:00:46.143065+0000","last_undegraded":"2024-09-11T08:00:46.143065+0000","last_fullsized":"2024-09-11T08:00:46.143065+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:22:51.854452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144724+0000","last_change":"2024-09-11T08:00:38.808521+0000","last_active":"2024-09-11T08:00:46.144724+0000","last_peered":"2024-09-11T08:00:46.144724+0000","last_clean":"2024-09-11T08:00:46.144724+0000","last_became_active":"2024-09-11T08:00:38.807979+0000","last_became_peered":"2024-09-11T08:00:38.807979+0000","last_unstale":"2024-09-11T08:00:46.144724+0000","last_undegraded":"2024-09-11T08:00:46.144724+0000","last_fullsized":"2024-09-11T08:00:46.144724+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:32:57.633567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.145321+0000","last_change":"2024-09-11T08:00:36.805492+0000","last_active":"2024-09-11T08:00:46.145321+0000","last_peered":"2024-09-11T08:00:46.145321+0000","last_clean":"2024-09-11T08:00:46.145321+0000","last_became_active":"2024-09-11T08:00:36.805216+0000","last_became_peered":"2024-09-11T08:00:36.805216+0000","last_unstale":"2024-09-11T08:00:46.145321+0000","last_undegraded":"2024-09-11T08:00:46.145321+0000","last_fullsized":"2024-09-11T08:00:46.145321+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:33:03.369780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:01:55.144540+0000","last_change":"2024-09-11T08:00:40.840792+0000","last_active":"2024-09-11T08:01:55.144540+0000","last_peered":"2024-09-11T08:01:55.144540+0000","last_clean":"2024-09-11T08:01:55.144540+0000","last_became_active":"2024-09-11T08:00:40.840525+0000","last_became_peered":"2024-09-11T08:00:40.840525+0000","last_unstale":"2024-09-11T08:01:55.144540+0000","last_undegraded":"2024-09-11T08:01:55.144540+0000","last_fullsized":"2024-09-11T08:01:55.144540+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:41:58.906478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143158+0000","last_change":"2024-09-11T08:00:42.838006+0000","last_active":"2024-09-11T08:00:46.143158+0000","last_peered":"2024-09-11T08:00:46.143158+0000","last_clean":"2024-09-11T08:00:46.143158+0000","last_became_active":"2024-09-11T08:00:42.837727+0000","last_became_peered":"2024-09-11T08:00:42.837727+0000","last_unstale":"2024-09-11T08:00:46.143158+0000","last_undegraded":"2024-09-11T08:00:46.143158+0000","last_fullsized":"2024-09-11T08:00:46.143158+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:58:16.999026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"59'28","reported_seq":81,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:01:55.144567+0000","last_change":"2024-09-11T08:00:38.819419+0000","last_active":"2024-09-11T08:01:55.144567+0000","last_peered":"2024-09-11T08:01:55.144567+0000","last_clean":"2024-09-11T08:01:55.144567+0000","last_became_active":"2024-09-11T08:00:38.818939+0000","last_became_peered":"2024-09-11T08:00:38.818939+0000","last_unstale":"2024-09-11T08:01:55.144567+0000","last_undegraded":"2024-09-11T08:01:55.144567+0000","last_fullsized":"2024-09-11T08:01:55.144567+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:00:23.229784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227189+0000","last_change":"2024-09-11T08:00:36.799345+0000","last_active":"2024-09-11T08:00:46.227189+0000","last_peered":"2024-09-11T08:00:46.227189+0000","last_clean":"2024-09-11T08:00:46.227189+0000","last_became_active":"2024-09-11T08:00:36.799090+0000","last_became_peered":"2024-09-11T08:00:36.799090+0000","last_unstale":"2024-09-11T08:00:46.227189+0000","last_undegraded":"2024-09-11T08:00:46.227189+0000","last_fullsized":"2024-09-11T08:00:46.227189+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:57:00.554977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143668+0000","last_change":"2024-09-11T08:00:40.837484+0000","last_active":"2024-09-11T08:00:46.143668+0000","last_peered":"2024-09-11T08:00:46.143668+0000","last_clean":"2024-09-11T08:00:46.143668+0000","last_became_active":"2024-09-11T08:00:40.837112+0000","last_became_peered":"2024-09-11T08:00:40.837112+0000","last_unstale":"2024-09-11T08:00:46.143668+0000","last_undegraded":"2024-09-11T08:00:46.143668+0000","last_fullsized":"2024-09-11T08:00:46.143668+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:50:31.274414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222628+0000","last_change":"2024-09-11T08:00:42.831942+0000","last_active":"2024-09-11T08:00:46.222628+0000","last_peered":"2024-09-11T08:00:46.222628+0000","last_clean":"2024-09-11T08:00:46.222628+0000","last_became_active":"2024-09-11T08:00:42.831669+0000","last_became_peered":"2024-09-11T08:00:42.831669+0000","last_unstale":"2024-09-11T08:00:46.222628+0000","last_undegraded":"2024-09-11T08:00:46.222628+0000","last_fullsized":"2024-09-11T08:00:46.222628+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:09:59.680384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"59'13","reported_seq":43,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144565+0000","last_change":"2024-09-11T08:00:38.817893+0000","last_active":"2024-09-11T08:00:46.144565+0000","last_peered":"2024-09-11T08:00:46.144565+0000","last_clean":"2024-09-11T08:00:46.144565+0000","last_became_active":"2024-09-11T08:00:38.817550+0000","last_became_peered":"2024-09-11T08:00:38.817550+0000","last_unstale":"2024-09-11T08:00:46.144565+0000","last_undegraded":"2024-09-11T08:00:46.144565+0000","last_fullsized":"2024-09-11T08:00:46.144565+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:40:28.158387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"60'5","reported_seq":102,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:01:58.403771+0000","last_change":"2024-09-11T08:00:40.823639+0000","last_active":"2024-09-11T08:01:58.403771+0000","last_peered":"2024-09-11T08:01:58.403771+0000","last_clean":"2024-09-11T08:01:58.403771+0000","last_became_active":"2024-09-11T08:00:36.796900+0000","last_became_peered":"2024-09-11T08:00:36.796900+0000","last_unstale":"2024-09-11T08:01:58.403771+0000","last_undegraded":"2024-09-11T08:01:58.403771+0000","last_fullsized":"2024-09-11T08:01:58.403771+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:26:47.119529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039650379999999997,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226928+0000","last_change":"2024-09-11T08:00:36.806792+0000","last_active":"2024-09-11T08:00:46.226928+0000","last_peered":"2024-09-11T08:00:46.226928+0000","last_clean":"2024-09-11T08:00:46.226928+0000","last_became_active":"2024-09-11T08:00:36.806365+0000","last_became_peered":"2024-09-11T08:00:36.806365+0000","last_unstale":"2024-09-11T08:00:46.226928+0000","last_undegraded":"2024-09-11T08:00:46.226928+0000","last_fullsized":"2024-09-11T08:00:46.226928+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:39:38.024552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227292+0000","last_change":"2024-09-11T08:00:40.825697+0000","last_active":"2024-09-11T08:00:46.227292+0000","last_peered":"2024-09-11T08:00:46.227292+0000","last_clean":"2024-09-11T08:00:46.227292+0000","last_became_active":"2024-09-11T08:00:40.825391+0000","last_became_peered":"2024-09-11T08:00:40.825391+0000","last_unstale":"2024-09-11T08:00:46.227292+0000","last_undegraded":"2024-09-11T08:00:46.227292+0000","last_fullsized":"2024-09-11T08:00:46.227292+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:19:41.889688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227383+0000","last_change":"2024-09-11T08:00:43.068381+0000","last_active":"2024-09-11T08:00:46.227383+0000","last_peered":"2024-09-11T08:00:46.227383+0000","last_clean":"2024-09-11T08:00:46.227383+0000","last_became_active":"2024-09-11T08:00:43.068092+0000","last_became_peered":"2024-09-11T08:00:43.068092+0000","last_unstale":"2024-09-11T08:00:46.227383+0000","last_undegraded":"2024-09-11T08:00:46.227383+0000","last_fullsized":"2024-09-11T08:00:46.227383+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:01.390541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"59'12","reported_seq":34,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144838+0000","last_change":"2024-09-11T08:00:38.820878+0000","last_active":"2024-09-11T08:00:46.144838+0000","last_peered":"2024-09-11T08:00:46.144838+0000","last_clean":"2024-09-11T08:00:46.144838+0000","last_became_active":"2024-09-11T08:00:38.820561+0000","last_became_peered":"2024-09-11T08:00:38.820561+0000","last_unstale":"2024-09-11T08:00:46.144838+0000","last_undegraded":"2024-09-11T08:00:46.144838+0000","last_fullsized":"2024-09-11T08:00:46.144838+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:43:02.244072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"54'1","reported_seq":24,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144162+0000","last_change":"2024-09-11T08:00:40.826552+0000","last_active":"2024-09-11T08:00:46.144162+0000","last_peered":"2024-09-11T08:00:46.144162+0000","last_clean":"2024-09-11T08:00:46.144162+0000","last_became_active":"2024-09-11T08:00:36.806294+0000","last_became_peered":"2024-09-11T08:00:36.806294+0000","last_unstale":"2024-09-11T08:00:46.144162+0000","last_undegraded":"2024-09-11T08:00:46.144162+0000","last_fullsized":"2024-09-11T08:00:46.144162+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:38:39.602628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061749899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226758+0000","last_change":"2024-09-11T08:00:36.809229+0000","last_active":"2024-09-11T08:00:46.226758+0000","last_peered":"2024-09-11T08:00:46.226758+0000","last_clean":"2024-09-11T08:00:46.226758+0000","last_became_active":"2024-09-11T08:00:36.808915+0000","last_became_peered":"2024-09-11T08:00:36.808915+0000","last_unstale":"2024-09-11T08:00:46.226758+0000","last_undegraded":"2024-09-11T08:00:46.226758+0000","last_fullsized":"2024-09-11T08:00:46.226758+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:42:22.872114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.228060+0000","last_change":"2024-09-11T08:00:40.839326+0000","last_active":"2024-09-11T08:00:46.228060+0000","last_peered":"2024-09-11T08:00:46.228060+0000","last_clean":"2024-09-11T08:00:46.228060+0000","last_became_active":"2024-09-11T08:00:40.838821+0000","last_became_peered":"2024-09-11T08:00:40.838821+0000","last_unstale":"2024-09-11T08:00:46.228060+0000","last_undegraded":"2024-09-11T08:00:46.228060+0000","last_fullsized":"2024-09-11T08:00:46.228060+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:27:19.392496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.145096+0000","last_change":"2024-09-11T08:00:42.830924+0000","last_active":"2024-09-11T08:00:46.145096+0000","last_peered":"2024-09-11T08:00:46.145096+0000","last_clean":"2024-09-11T08:00:46.145096+0000","last_became_active":"2024-09-11T08:00:42.830587+0000","last_became_peered":"2024-09-11T08:00:42.830587+0000","last_unstale":"2024-09-11T08:00:46.145096+0000","last_undegraded":"2024-09-11T08:00:46.145096+0000","last_fullsized":"2024-09-11T08:00:46.145096+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:10:34.296714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"59'14","reported_seq":38,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.200423+0000","last_change":"2024-09-11T08:00:39.123611+0000","last_active":"2024-09-11T08:00:46.200423+0000","last_peered":"2024-09-11T08:00:46.200423+0000","last_clean":"2024-09-11T08:00:46.200423+0000","last_became_active":"2024-09-11T08:00:39.123207+0000","last_became_peered":"2024-09-11T08:00:39.123207+0000","last_unstale":"2024-09-11T08:00:46.200423+0000","last_undegraded":"2024-09-11T08:00:46.200423+0000","last_fullsized":"2024-09-11T08:00:46.200423+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:08:50.532757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'2","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222219+0000","last_change":"2024-09-11T08:00:40.826568+0000","last_active":"2024-09-11T08:00:46.222219+0000","last_peered":"2024-09-11T08:00:46.222219+0000","last_clean":"2024-09-11T08:00:46.222219+0000","last_became_active":"2024-09-11T08:00:36.797923+0000","last_became_peered":"2024-09-11T08:00:36.797923+0000","last_unstale":"2024-09-11T08:00:46.222219+0000","last_undegraded":"2024-09-11T08:00:46.222219+0000","last_fullsized":"2024-09-11T08:00:46.222219+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:09:52.150614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028090279999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227911+0000","last_change":"2024-09-11T08:00:36.801632+0000","last_active":"2024-09-11T08:00:46.227911+0000","last_peered":"2024-09-11T08:00:46.227911+0000","last_clean":"2024-09-11T08:00:46.227911+0000","last_became_active":"2024-09-11T08:00:36.801375+0000","last_became_peered":"2024-09-11T08:00:36.801375+0000","last_unstale":"2024-09-11T08:00:46.227911+0000","last_undegraded":"2024-09-11T08:00:46.227911+0000","last_fullsized":"2024-09-11T08:00:46.227911+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:25:26.929458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"61'79","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:01:16.379109+0000","last_change":"2024-09-11T08:00:25.551556+0000","last_active":"2024-09-11T08:01:16.379109+0000","last_peered":"2024-09-11T08:01:16.379109+0000","last_clean":"2024-09-11T08:01:16.379109+0000","last_became_active":"2024-09-11T08:00:25.547109+0000","last_became_peered":"2024-09-11T08:00:25.547109+0000","last_unstale":"2024-09-11T08:01:16.379109+0000","last_undegraded":"2024-09-11T08:01:16.379109+0000","last_fullsized":"2024-09-11T08:01:16.379109+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:58:30.437120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:58:30.437120+0000","last_clean_scrub_stamp":"2024-09-11T07:58:30.437120+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:18.497543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227796+0000","last_change":"2024-09-11T08:00:40.831304+0000","last_active":"2024-09-11T08:00:46.227796+0000","last_peered":"2024-09-11T08:00:46.227796+0000","last_clean":"2024-09-11T08:00:46.227796+0000","last_became_active":"2024-09-11T08:00:40.831015+0000","last_became_peered":"2024-09-11T08:00:40.831015+0000","last_unstale":"2024-09-11T08:00:46.227796+0000","last_undegraded":"2024-09-11T08:00:46.227796+0000","last_fullsized":"2024-09-11T08:00:46.227796+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:48.050004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.228130+0000","last_change":"2024-09-11T08:00:42.832223+0000","last_active":"2024-09-11T08:00:46.228130+0000","last_peered":"2024-09-11T08:00:46.228130+0000","last_clean":"2024-09-11T08:00:46.228130+0000","last_became_active":"2024-09-11T08:00:42.831908+0000","last_became_peered":"2024-09-11T08:00:42.831908+0000","last_unstale":"2024-09-11T08:00:46.228130+0000","last_undegraded":"2024-09-11T08:00:46.228130+0000","last_fullsized":"2024-09-11T08:00:46.228130+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:13:19.334798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"59'19","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.200503+0000","last_change":"2024-09-11T08:00:39.123880+0000","last_active":"2024-09-11T08:00:46.200503+0000","last_peered":"2024-09-11T08:00:46.200503+0000","last_clean":"2024-09-11T08:00:46.200503+0000","last_became_active":"2024-09-11T08:00:39.123540+0000","last_became_peered":"2024-09-11T08:00:39.123540+0000","last_unstale":"2024-09-11T08:00:46.200503+0000","last_undegraded":"2024-09-11T08:00:46.200503+0000","last_fullsized":"2024-09-11T08:00:46.200503+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:55:10.411428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227377+0000","last_change":"2024-09-11T08:00:36.803359+0000","last_active":"2024-09-11T08:00:46.227377+0000","last_peered":"2024-09-11T08:00:46.227377+0000","last_clean":"2024-09-11T08:00:46.227377+0000","last_became_active":"2024-09-11T08:00:36.803101+0000","last_became_peered":"2024-09-11T08:00:36.803101+0000","last_unstale":"2024-09-11T08:00:46.227377+0000","last_undegraded":"2024-09-11T08:00:46.227377+0000","last_fullsized":"2024-09-11T08:00:46.227377+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:28:45.051668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227023+0000","last_change":"2024-09-11T08:00:40.830973+0000","last_active":"2024-09-11T08:00:46.227023+0000","last_peered":"2024-09-11T08:00:46.227023+0000","last_clean":"2024-09-11T08:00:46.227023+0000","last_became_active":"2024-09-11T08:00:40.830672+0000","last_became_peered":"2024-09-11T08:00:40.830672+0000","last_unstale":"2024-09-11T08:00:46.227023+0000","last_undegraded":"2024-09-11T08:00:46.227023+0000","last_fullsized":"2024-09-11T08:00:46.227023+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:49:06.396532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226795+0000","last_change":"2024-09-11T08:00:42.834584+0000","last_active":"2024-09-11T08:00:46.226795+0000","last_peered":"2024-09-11T08:00:46.226795+0000","last_clean":"2024-09-11T08:00:46.226795+0000","last_became_active":"2024-09-11T08:00:42.834299+0000","last_became_peered":"2024-09-11T08:00:42.834299+0000","last_unstale":"2024-09-11T08:00:46.226795+0000","last_undegraded":"2024-09-11T08:00:46.226795+0000","last_fullsized":"2024-09-11T08:00:46.226795+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:26:51.062757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144569+0000","last_change":"2024-09-11T08:00:38.819310+0000","last_active":"2024-09-11T08:00:46.144569+0000","last_peered":"2024-09-11T08:00:46.144569+0000","last_clean":"2024-09-11T08:00:46.144569+0000","last_became_active":"2024-09-11T08:00:38.818756+0000","last_became_peered":"2024-09-11T08:00:38.818756+0000","last_unstale":"2024-09-11T08:00:46.144569+0000","last_undegraded":"2024-09-11T08:00:46.144569+0000","last_fullsized":"2024-09-11T08:00:46.144569+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:22:22.696900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.145099+0000","last_change":"2024-09-11T08:00:36.797701+0000","last_active":"2024-09-11T08:00:46.145099+0000","last_peered":"2024-09-11T08:00:46.145099+0000","last_clean":"2024-09-11T08:00:46.145099+0000","last_became_active":"2024-09-11T08:00:36.797454+0000","last_became_peered":"2024-09-11T08:00:36.797454+0000","last_unstale":"2024-09-11T08:00:46.145099+0000","last_undegraded":"2024-09-11T08:00:46.145099+0000","last_fullsized":"2024-09-11T08:00:46.145099+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:18:53.934285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226928+0000","last_change":"2024-09-11T08:00:40.833591+0000","last_active":"2024-09-11T08:00:46.226928+0000","last_peered":"2024-09-11T08:00:46.226928+0000","last_clean":"2024-09-11T08:00:46.226928+0000","last_became_active":"2024-09-11T08:00:40.833225+0000","last_became_peered":"2024-09-11T08:00:40.833225+0000","last_unstale":"2024-09-11T08:00:46.226928+0000","last_undegraded":"2024-09-11T08:00:46.226928+0000","last_fullsized":"2024-09-11T08:00:46.226928+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:10:49.193831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144003+0000","last_change":"2024-09-11T08:00:42.833928+0000","last_active":"2024-09-11T08:00:46.144003+0000","last_peered":"2024-09-11T08:00:46.144003+0000","last_clean":"2024-09-11T08:00:46.144003+0000","last_became_active":"2024-09-11T08:00:42.833641+0000","last_became_peered":"2024-09-11T08:00:42.833641+0000","last_unstale":"2024-09-11T08:00:46.144003+0000","last_undegraded":"2024-09-11T08:00:46.144003+0000","last_fullsized":"2024-09-11T08:00:46.144003+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:24:33.720118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226927+0000","last_change":"2024-09-11T08:00:39.123656+0000","last_active":"2024-09-11T08:00:46.226927+0000","last_peered":"2024-09-11T08:00:46.226927+0000","last_clean":"2024-09-11T08:00:46.226927+0000","last_became_active":"2024-09-11T08:00:39.123347+0000","last_became_peered":"2024-09-11T08:00:39.123347+0000","last_unstale":"2024-09-11T08:00:46.226927+0000","last_undegraded":"2024-09-11T08:00:46.226927+0000","last_fullsized":"2024-09-11T08:00:46.226927+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:40:44.909896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227399+0000","last_change":"2024-09-11T08:00:36.801797+0000","last_active":"2024-09-11T08:00:46.227399+0000","last_peered":"2024-09-11T08:00:46.227399+0000","last_clean":"2024-09-11T08:00:46.227399+0000","last_became_active":"2024-09-11T08:00:36.801515+0000","last_became_peered":"2024-09-11T08:00:36.801515+0000","last_unstale":"2024-09-11T08:00:46.227399+0000","last_undegraded":"2024-09-11T08:00:46.227399+0000","last_fullsized":"2024-09-11T08:00:46.227399+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:27.022074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"59'8","reported_seq":42,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:01:55.144605+0000","last_change":"2024-09-11T08:00:40.829863+0000","last_active":"2024-09-11T08:01:55.144605+0000","last_peered":"2024-09-11T08:01:55.144605+0000","last_clean":"2024-09-11T08:01:55.144605+0000","last_became_active":"2024-09-11T08:00:40.829265+0000","last_became_peered":"2024-09-11T08:00:40.829265+0000","last_unstale":"2024-09-11T08:01:55.144605+0000","last_undegraded":"2024-09-11T08:01:55.144605+0000","last_fullsized":"2024-09-11T08:01:55.144605+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:36.438007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226671+0000","last_change":"2024-09-11T08:00:43.068277+0000","last_active":"2024-09-11T08:00:46.226671+0000","last_peered":"2024-09-11T08:00:46.226671+0000","last_clean":"2024-09-11T08:00:46.226671+0000","last_became_active":"2024-09-11T08:00:43.067892+0000","last_became_peered":"2024-09-11T08:00:43.067892+0000","last_unstale":"2024-09-11T08:00:46.226671+0000","last_undegraded":"2024-09-11T08:00:46.226671+0000","last_fullsized":"2024-09-11T08:00:46.226671+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:24:09.535763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.141611+0000","last_change":"2024-09-11T08:00:38.815899+0000","last_active":"2024-09-11T08:00:46.141611+0000","last_peered":"2024-09-11T08:00:46.141611+0000","last_clean":"2024-09-11T08:00:46.141611+0000","last_became_active":"2024-09-11T08:00:38.815498+0000","last_became_peered":"2024-09-11T08:00:38.815498+0000","last_unstale":"2024-09-11T08:00:46.141611+0000","last_undegraded":"2024-09-11T08:00:46.141611+0000","last_fullsized":"2024-09-11T08:00:46.141611+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:50:21.463132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"52'2","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.142502+0000","last_change":"2024-09-11T08:00:36.809569+0000","last_active":"2024-09-11T08:00:46.142502+0000","last_peered":"2024-09-11T08:00:46.142502+0000","last_clean":"2024-09-11T08:00:46.142502+0000","last_became_active":"2024-09-11T08:00:36.809000+0000","last_became_peered":"2024-09-11T08:00:36.809000+0000","last_unstale":"2024-09-11T08:00:46.142502+0000","last_undegraded":"2024-09-11T08:00:46.142502+0000","last_fullsized":"2024-09-11T08:00:46.142502+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:36:42.736820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226864+0000","last_change":"2024-09-11T08:00:40.836191+0000","last_active":"2024-09-11T08:00:46.226864+0000","last_peered":"2024-09-11T08:00:46.226864+0000","last_clean":"2024-09-11T08:00:46.226864+0000","last_became_active":"2024-09-11T08:00:40.835918+0000","last_became_peered":"2024-09-11T08:00:40.835918+0000","last_unstale":"2024-09-11T08:00:46.226864+0000","last_undegraded":"2024-09-11T08:00:46.226864+0000","last_fullsized":"2024-09-11T08:00:46.226864+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:02:13.637003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222170+0000","last_change":"2024-09-11T08:00:42.829063+0000","last_active":"2024-09-11T08:00:46.222170+0000","last_peered":"2024-09-11T08:00:46.222170+0000","last_clean":"2024-09-11T08:00:46.222170+0000","last_became_active":"2024-09-11T08:00:42.828748+0000","last_became_peered":"2024-09-11T08:00:42.828748+0000","last_unstale":"2024-09-11T08:00:46.222170+0000","last_undegraded":"2024-09-11T08:00:46.222170+0000","last_fullsized":"2024-09-11T08:00:46.222170+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:55:07.283890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144763+0000","last_change":"2024-09-11T08:00:39.123200+0000","last_active":"2024-09-11T08:00:46.144763+0000","last_peered":"2024-09-11T08:00:46.144763+0000","last_clean":"2024-09-11T08:00:46.144763+0000","last_became_active":"2024-09-11T08:00:39.122900+0000","last_became_peered":"2024-09-11T08:00:39.122900+0000","last_unstale":"2024-09-11T08:00:46.144763+0000","last_undegraded":"2024-09-11T08:00:46.144763+0000","last_fullsized":"2024-09-11T08:00:46.144763+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:26:51.490458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"52'1","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226944+0000","last_change":"2024-09-11T08:00:36.808502+0000","last_active":"2024-09-11T08:00:46.226944+0000","last_peered":"2024-09-11T08:00:46.226944+0000","last_clean":"2024-09-11T08:00:46.226944+0000","last_became_active":"2024-09-11T08:00:36.808215+0000","last_became_peered":"2024-09-11T08:00:36.808215+0000","last_unstale":"2024-09-11T08:00:46.226944+0000","last_undegraded":"2024-09-11T08:00:46.226944+0000","last_fullsized":"2024-09-11T08:00:46.226944+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:24:49.381951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222774+0000","last_change":"2024-09-11T08:00:40.831998+0000","last_active":"2024-09-11T08:00:46.222774+0000","last_peered":"2024-09-11T08:00:46.222774+0000","last_clean":"2024-09-11T08:00:46.222774+0000","last_became_active":"2024-09-11T08:00:40.831689+0000","last_became_peered":"2024-09-11T08:00:40.831689+0000","last_unstale":"2024-09-11T08:00:46.222774+0000","last_undegraded":"2024-09-11T08:00:46.222774+0000","last_fullsized":"2024-09-11T08:00:46.222774+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:29:51.364092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227066+0000","last_change":"2024-09-11T08:00:42.837761+0000","last_active":"2024-09-11T08:00:46.227066+0000","last_peered":"2024-09-11T08:00:46.227066+0000","last_clean":"2024-09-11T08:00:46.227066+0000","last_became_active":"2024-09-11T08:00:42.837503+0000","last_became_peered":"2024-09-11T08:00:42.837503+0000","last_unstale":"2024-09-11T08:00:46.227066+0000","last_undegraded":"2024-09-11T08:00:46.227066+0000","last_fullsized":"2024-09-11T08:00:46.227066+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:32:09.094037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.200431+0000","last_change":"2024-09-11T08:00:36.797052+0000","last_active":"2024-09-11T08:00:46.200431+0000","last_peered":"2024-09-11T08:00:46.200431+0000","last_clean":"2024-09-11T08:00:46.200431+0000","last_became_active":"2024-09-11T08:00:36.796792+0000","last_became_peered":"2024-09-11T08:00:36.796792+0000","last_unstale":"2024-09-11T08:00:46.200431+0000","last_undegraded":"2024-09-11T08:00:46.200431+0000","last_fullsized":"2024-09-11T08:00:46.200431+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:38:50.114641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"59'6","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222862+0000","last_change":"2024-09-11T08:00:39.123764+0000","last_active":"2024-09-11T08:00:46.222862+0000","last_peered":"2024-09-11T08:00:46.222862+0000","last_clean":"2024-09-11T08:00:46.222862+0000","last_became_active":"2024-09-11T08:00:39.123515+0000","last_became_peered":"2024-09-11T08:00:39.123515+0000","last_unstale":"2024-09-11T08:00:46.222862+0000","last_undegraded":"2024-09-11T08:00:46.222862+0000","last_fullsized":"2024-09-11T08:00:46.222862+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:32:48.067610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227937+0000","last_change":"2024-09-11T08:00:40.838777+0000","last_active":"2024-09-11T08:00:46.227937+0000","last_peered":"2024-09-11T08:00:46.227937+0000","last_clean":"2024-09-11T08:00:46.227937+0000","last_became_active":"2024-09-11T08:00:40.838445+0000","last_became_peered":"2024-09-11T08:00:40.838445+0000","last_unstale":"2024-09-11T08:00:46.227937+0000","last_undegraded":"2024-09-11T08:00:46.227937+0000","last_fullsized":"2024-09-11T08:00:46.227937+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:06:52.443015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227301+0000","last_change":"2024-09-11T08:00:43.068823+0000","last_active":"2024-09-11T08:00:46.227301+0000","last_peered":"2024-09-11T08:00:46.227301+0000","last_clean":"2024-09-11T08:00:46.227301+0000","last_became_active":"2024-09-11T08:00:43.068528+0000","last_became_peered":"2024-09-11T08:00:43.068528+0000","last_unstale":"2024-09-11T08:00:46.227301+0000","last_undegraded":"2024-09-11T08:00:46.227301+0000","last_fullsized":"2024-09-11T08:00:46.227301+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:38:40.760170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222731+0000","last_change":"2024-09-11T08:00:39.123193+0000","last_active":"2024-09-11T08:00:46.222731+0000","last_peered":"2024-09-11T08:00:46.222731+0000","last_clean":"2024-09-11T08:00:46.222731+0000","last_became_active":"2024-09-11T08:00:39.122917+0000","last_became_peered":"2024-09-11T08:00:39.122917+0000","last_unstale":"2024-09-11T08:00:46.222731+0000","last_undegraded":"2024-09-11T08:00:46.222731+0000","last_fullsized":"2024-09-11T08:00:46.222731+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:08:27.508033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227123+0000","last_change":"2024-09-11T08:00:36.802610+0000","last_active":"2024-09-11T08:00:46.227123+0000","last_peered":"2024-09-11T08:00:46.227123+0000","last_clean":"2024-09-11T08:00:46.227123+0000","last_became_active":"2024-09-11T08:00:36.802368+0000","last_became_peered":"2024-09-11T08:00:36.802368+0000","last_unstale":"2024-09-11T08:00:46.227123+0000","last_undegraded":"2024-09-11T08:00:46.227123+0000","last_fullsized":"2024-09-11T08:00:46.227123+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:04:07.662727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:01:55.144614+0000","last_change":"2024-09-11T08:00:40.836077+0000","last_active":"2024-09-11T08:01:55.144614+0000","last_peered":"2024-09-11T08:01:55.144614+0000","last_clean":"2024-09-11T08:01:55.144614+0000","last_became_active":"2024-09-11T08:00:40.835757+0000","last_became_peered":"2024-09-11T08:00:40.835757+0000","last_unstale":"2024-09-11T08:01:55.144614+0000","last_undegraded":"2024-09-11T08:01:55.144614+0000","last_fullsized":"2024-09-11T08:01:55.144614+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:52:10.537609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143163+0000","last_change":"2024-09-11T08:00:42.837104+0000","last_active":"2024-09-11T08:00:46.143163+0000","last_peered":"2024-09-11T08:00:46.143163+0000","last_clean":"2024-09-11T08:00:46.143163+0000","last_became_active":"2024-09-11T08:00:42.836669+0000","last_became_peered":"2024-09-11T08:00:42.836669+0000","last_unstale":"2024-09-11T08:00:46.143163+0000","last_undegraded":"2024-09-11T08:00:46.143163+0000","last_fullsized":"2024-09-11T08:00:46.143163+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:29.048875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.228158+0000","last_change":"2024-09-11T08:00:39.123814+0000","last_active":"2024-09-11T08:00:46.228158+0000","last_peered":"2024-09-11T08:00:46.228158+0000","last_clean":"2024-09-11T08:00:46.228158+0000","last_became_active":"2024-09-11T08:00:39.123464+0000","last_became_peered":"2024-09-11T08:00:39.123464+0000","last_unstale":"2024-09-11T08:00:46.228158+0000","last_undegraded":"2024-09-11T08:00:46.228158+0000","last_fullsized":"2024-09-11T08:00:46.228158+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:33:48.390737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.145203+0000","last_change":"2024-09-11T08:00:36.805490+0000","last_active":"2024-09-11T08:00:46.145203+0000","last_peered":"2024-09-11T08:00:46.145203+0000","last_clean":"2024-09-11T08:00:46.145203+0000","last_became_active":"2024-09-11T08:00:36.804674+0000","last_became_peered":"2024-09-11T08:00:36.804674+0000","last_unstale":"2024-09-11T08:00:46.145203+0000","last_undegraded":"2024-09-11T08:00:46.145203+0000","last_fullsized":"2024-09-11T08:00:46.145203+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:10.808200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:01:55.144773+0000","last_change":"2024-09-11T08:00:40.827676+0000","last_active":"2024-09-11T08:01:55.144773+0000","last_peered":"2024-09-11T08:01:55.144773+0000","last_clean":"2024-09-11T08:01:55.144773+0000","last_became_active":"2024-09-11T08:00:40.827324+0000","last_became_peered":"2024-09-11T08:00:40.827324+0000","last_unstale":"2024-09-11T08:01:55.144773+0000","last_undegraded":"2024-09-11T08:01:55.144773+0000","last_fullsized":"2024-09-11T08:01:55.144773+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:07:00.275085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.142278+0000","last_change":"2024-09-11T08:00:42.829729+0000","last_active":"2024-09-11T08:00:46.142278+0000","last_peered":"2024-09-11T08:00:46.142278+0000","last_clean":"2024-09-11T08:00:46.142278+0000","last_became_active":"2024-09-11T08:00:42.829363+0000","last_became_peered":"2024-09-11T08:00:42.829363+0000","last_unstale":"2024-09-11T08:00:46.142278+0000","last_undegraded":"2024-09-11T08:00:46.142278+0000","last_fullsized":"2024-09-11T08:00:46.142278+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:11:33.273548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144622+0000","last_change":"2024-09-11T08:00:38.811353+0000","last_active":"2024-09-11T08:00:46.144622+0000","last_peered":"2024-09-11T08:00:46.144622+0000","last_clean":"2024-09-11T08:00:46.144622+0000","last_became_active":"2024-09-11T08:00:38.811069+0000","last_became_peered":"2024-09-11T08:00:38.811069+0000","last_unstale":"2024-09-11T08:00:46.144622+0000","last_undegraded":"2024-09-11T08:00:46.144622+0000","last_fullsized":"2024-09-11T08:00:46.144622+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:33.934806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.200421+0000","last_change":"2024-09-11T08:00:36.803793+0000","last_active":"2024-09-11T08:00:46.200421+0000","last_peered":"2024-09-11T08:00:46.200421+0000","last_clean":"2024-09-11T08:00:46.200421+0000","last_became_active":"2024-09-11T08:00:36.803536+0000","last_became_peered":"2024-09-11T08:00:36.803536+0000","last_unstale":"2024-09-11T08:00:46.200421+0000","last_undegraded":"2024-09-11T08:00:46.200421+0000","last_fullsized":"2024-09-11T08:00:46.200421+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:20:44.229786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222245+0000","last_change":"2024-09-11T08:00:40.831179+0000","last_active":"2024-09-11T08:00:46.222245+0000","last_peered":"2024-09-11T08:00:46.222245+0000","last_clean":"2024-09-11T08:00:46.222245+0000","last_became_active":"2024-09-11T08:00:40.830817+0000","last_became_peered":"2024-09-11T08:00:40.830817+0000","last_unstale":"2024-09-11T08:00:46.222245+0000","last_undegraded":"2024-09-11T08:00:46.222245+0000","last_fullsized":"2024-09-11T08:00:46.222245+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:32:09.861168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143065+0000","last_change":"2024-09-11T08:00:42.835761+0000","last_active":"2024-09-11T08:00:46.143065+0000","last_peered":"2024-09-11T08:00:46.143065+0000","last_clean":"2024-09-11T08:00:46.143065+0000","last_became_active":"2024-09-11T08:00:42.835437+0000","last_became_peered":"2024-09-11T08:00:42.835437+0000","last_unstale":"2024-09-11T08:00:46.143065+0000","last_undegraded":"2024-09-11T08:00:46.143065+0000","last_fullsized":"2024-09-11T08:00:46.143065+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:04:24.902266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.141966+0000","last_change":"2024-09-11T08:00:38.807375+0000","last_active":"2024-09-11T08:00:46.141966+0000","last_peered":"2024-09-11T08:00:46.141966+0000","last_clean":"2024-09-11T08:00:46.141966+0000","last_became_active":"2024-09-11T08:00:38.806989+0000","last_became_peered":"2024-09-11T08:00:38.806989+0000","last_unstale":"2024-09-11T08:00:46.141966+0000","last_undegraded":"2024-09-11T08:00:46.141966+0000","last_fullsized":"2024-09-11T08:00:46.141966+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:03:48.987467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.145098+0000","last_change":"2024-09-11T08:00:36.808566+0000","last_active":"2024-09-11T08:00:46.145098+0000","last_peered":"2024-09-11T08:00:46.145098+0000","last_clean":"2024-09-11T08:00:46.145098+0000","last_became_active":"2024-09-11T08:00:36.808007+0000","last_became_peered":"2024-09-11T08:00:36.808007+0000","last_unstale":"2024-09-11T08:00:46.145098+0000","last_undegraded":"2024-09-11T08:00:46.145098+0000","last_fullsized":"2024-09-11T08:00:46.145098+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:41:47.056360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143551+0000","last_change":"2024-09-11T08:00:40.830750+0000","last_active":"2024-09-11T08:00:46.143551+0000","last_peered":"2024-09-11T08:00:46.143551+0000","last_clean":"2024-09-11T08:00:46.143551+0000","last_became_active":"2024-09-11T08:00:40.830425+0000","last_became_peered":"2024-09-11T08:00:40.830425+0000","last_unstale":"2024-09-11T08:00:46.143551+0000","last_undegraded":"2024-09-11T08:00:46.143551+0000","last_fullsized":"2024-09-11T08:00:46.143551+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:45:40.883179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.221867+0000","last_change":"2024-09-11T08:00:42.836560+0000","last_active":"2024-09-11T08:00:46.221867+0000","last_peered":"2024-09-11T08:00:46.221867+0000","last_clean":"2024-09-11T08:00:46.221867+0000","last_became_active":"2024-09-11T08:00:42.836301+0000","last_became_peered":"2024-09-11T08:00:42.836301+0000","last_unstale":"2024-09-11T08:00:46.221867+0000","last_undegraded":"2024-09-11T08:00:46.221867+0000","last_fullsized":"2024-09-11T08:00:46.221867+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:34.297728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"59'4","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.221846+0000","last_change":"2024-09-11T08:00:38.820225+0000","last_active":"2024-09-11T08:00:46.221846+0000","last_peered":"2024-09-11T08:00:46.221846+0000","last_clean":"2024-09-11T08:00:46.221846+0000","last_became_active":"2024-09-11T08:00:38.819965+0000","last_became_peered":"2024-09-11T08:00:38.819965+0000","last_unstale":"2024-09-11T08:00:46.221846+0000","last_undegraded":"2024-09-11T08:00:46.221846+0000","last_fullsized":"2024-09-11T08:00:46.221846+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:35:12.218080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227996+0000","last_change":"2024-09-11T08:00:36.804017+0000","last_active":"2024-09-11T08:00:46.227996+0000","last_peered":"2024-09-11T08:00:46.227996+0000","last_clean":"2024-09-11T08:00:46.227996+0000","last_became_active":"2024-09-11T08:00:36.803760+0000","last_became_peered":"2024-09-11T08:00:36.803760+0000","last_unstale":"2024-09-11T08:00:46.227996+0000","last_undegraded":"2024-09-11T08:00:46.227996+0000","last_fullsized":"2024-09-11T08:00:46.227996+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:45:42.732251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.200230+0000","last_change":"2024-09-11T08:00:40.834099+0000","last_active":"2024-09-11T08:00:46.200230+0000","last_peered":"2024-09-11T08:00:46.200230+0000","last_clean":"2024-09-11T08:00:46.200230+0000","last_became_active":"2024-09-11T08:00:40.833782+0000","last_became_peered":"2024-09-11T08:00:40.833782+0000","last_unstale":"2024-09-11T08:00:46.200230+0000","last_undegraded":"2024-09-11T08:00:46.200230+0000","last_fullsized":"2024-09-11T08:00:46.200230+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:02:44.475232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226613+0000","last_change":"2024-09-11T08:00:42.834547+0000","last_active":"2024-09-11T08:00:46.226613+0000","last_peered":"2024-09-11T08:00:46.226613+0000","last_clean":"2024-09-11T08:00:46.226613+0000","last_became_active":"2024-09-11T08:00:42.834148+0000","last_became_peered":"2024-09-11T08:00:42.834148+0000","last_unstale":"2024-09-11T08:00:46.226613+0000","last_undegraded":"2024-09-11T08:00:46.226613+0000","last_fullsized":"2024-09-11T08:00:46.226613+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:05:16.241381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222031+0000","last_change":"2024-09-11T08:00:39.123191+0000","last_active":"2024-09-11T08:00:46.222031+0000","last_peered":"2024-09-11T08:00:46.222031+0000","last_clean":"2024-09-11T08:00:46.222031+0000","last_became_active":"2024-09-11T08:00:39.122951+0000","last_became_peered":"2024-09-11T08:00:39.122951+0000","last_unstale":"2024-09-11T08:00:46.222031+0000","last_undegraded":"2024-09-11T08:00:46.222031+0000","last_fullsized":"2024-09-11T08:00:46.222031+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:37:03.917433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.200724+0000","last_change":"2024-09-11T08:00:36.801849+0000","last_active":"2024-09-11T08:00:46.200724+0000","last_peered":"2024-09-11T08:00:46.200724+0000","last_clean":"2024-09-11T08:00:46.200724+0000","last_became_active":"2024-09-11T08:00:36.800948+0000","last_became_peered":"2024-09-11T08:00:36.800948+0000","last_unstale":"2024-09-11T08:00:46.200724+0000","last_undegraded":"2024-09-11T08:00:46.200724+0000","last_fullsized":"2024-09-11T08:00:46.200724+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:53:32.454700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226525+0000","last_change":"2024-09-11T08:00:40.834066+0000","last_active":"2024-09-11T08:00:46.226525+0000","last_peered":"2024-09-11T08:00:46.226525+0000","last_clean":"2024-09-11T08:00:46.226525+0000","last_became_active":"2024-09-11T08:00:40.833675+0000","last_became_peered":"2024-09-11T08:00:40.833675+0000","last_unstale":"2024-09-11T08:00:46.226525+0000","last_undegraded":"2024-09-11T08:00:46.226525+0000","last_fullsized":"2024-09-11T08:00:46.226525+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:23:18.250635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222102+0000","last_change":"2024-09-11T08:00:43.068452+0000","last_active":"2024-09-11T08:00:46.222102+0000","last_peered":"2024-09-11T08:00:46.222102+0000","last_clean":"2024-09-11T08:00:46.222102+0000","last_became_active":"2024-09-11T08:00:43.068135+0000","last_became_peered":"2024-09-11T08:00:43.068135+0000","last_unstale":"2024-09-11T08:00:46.222102+0000","last_undegraded":"2024-09-11T08:00:46.222102+0000","last_fullsized":"2024-09-11T08:00:46.222102+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:23.938820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227466+0000","last_change":"2024-09-11T08:00:42.835011+0000","last_active":"2024-09-11T08:00:46.227466+0000","last_peered":"2024-09-11T08:00:46.227466+0000","last_clean":"2024-09-11T08:00:46.227466+0000","last_became_active":"2024-09-11T08:00:42.834758+0000","last_became_peered":"2024-09-11T08:00:42.834758+0000","last_unstale":"2024-09-11T08:00:46.227466+0000","last_undegraded":"2024-09-11T08:00:46.227466+0000","last_fullsized":"2024-09-11T08:00:46.227466+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:01:07.211070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.226955+0000","last_change":"2024-09-11T08:00:36.801605+0000","last_active":"2024-09-11T08:00:46.226955+0000","last_peered":"2024-09-11T08:00:46.226955+0000","last_clean":"2024-09-11T08:00:46.226955+0000","last_became_active":"2024-09-11T08:00:36.801324+0000","last_became_peered":"2024-09-11T08:00:36.801324+0000","last_unstale":"2024-09-11T08:00:46.226955+0000","last_undegraded":"2024-09-11T08:00:46.226955+0000","last_fullsized":"2024-09-11T08:00:46.226955+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:53:36.775862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143732+0000","last_change":"2024-09-11T08:00:39.123777+0000","last_active":"2024-09-11T08:00:46.143732+0000","last_peered":"2024-09-11T08:00:46.143732+0000","last_clean":"2024-09-11T08:00:46.143732+0000","last_became_active":"2024-09-11T08:00:39.123477+0000","last_became_peered":"2024-09-11T08:00:39.123477+0000","last_unstale":"2024-09-11T08:00:46.143732+0000","last_undegraded":"2024-09-11T08:00:46.143732+0000","last_fullsized":"2024-09-11T08:00:46.143732+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:54:29.890493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"59'8","reported_seq":42,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T08:01:55.144675+0000","last_change":"2024-09-11T08:00:40.840322+0000","last_active":"2024-09-11T08:01:55.144675+0000","last_peered":"2024-09-11T08:01:55.144675+0000","last_clean":"2024-09-11T08:01:55.144675+0000","last_became_active":"2024-09-11T08:00:40.840008+0000","last_became_peered":"2024-09-11T08:00:40.840008+0000","last_unstale":"2024-09-11T08:01:55.144675+0000","last_undegraded":"2024-09-11T08:01:55.144675+0000","last_fullsized":"2024-09-11T08:01:55.144675+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:03.732737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143755+0000","last_change":"2024-09-11T08:00:42.831914+0000","last_active":"2024-09-11T08:00:46.143755+0000","last_peered":"2024-09-11T08:00:46.143755+0000","last_clean":"2024-09-11T08:00:46.143755+0000","last_became_active":"2024-09-11T08:00:42.831600+0000","last_became_peered":"2024-09-11T08:00:42.831600+0000","last_unstale":"2024-09-11T08:00:46.143755+0000","last_undegraded":"2024-09-11T08:00:46.143755+0000","last_fullsized":"2024-09-11T08:00:46.143755+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:14:02.421234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222508+0000","last_change":"2024-09-11T08:00:36.808458+0000","last_active":"2024-09-11T08:00:46.222508+0000","last_peered":"2024-09-11T08:00:46.222508+0000","last_clean":"2024-09-11T08:00:46.222508+0000","last_became_active":"2024-09-11T08:00:36.808188+0000","last_became_peered":"2024-09-11T08:00:36.808188+0000","last_unstale":"2024-09-11T08:00:46.222508+0000","last_undegraded":"2024-09-11T08:00:46.222508+0000","last_fullsized":"2024-09-11T08:00:46.222508+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:52:58.705799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.200654+0000","last_change":"2024-09-11T08:00:38.815295+0000","last_active":"2024-09-11T08:00:46.200654+0000","last_peered":"2024-09-11T08:00:46.200654+0000","last_clean":"2024-09-11T08:00:46.200654+0000","last_became_active":"2024-09-11T08:00:38.814976+0000","last_became_peered":"2024-09-11T08:00:38.814976+0000","last_unstale":"2024-09-11T08:00:46.200654+0000","last_undegraded":"2024-09-11T08:00:46.200654+0000","last_fullsized":"2024-09-11T08:00:46.200654+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:39:32.310184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.143324+0000","last_change":"2024-09-11T08:00:40.834883+0000","last_active":"2024-09-11T08:00:46.143324+0000","last_peered":"2024-09-11T08:00:46.143324+0000","last_clean":"2024-09-11T08:00:46.143324+0000","last_became_active":"2024-09-11T08:00:40.834570+0000","last_became_peered":"2024-09-11T08:00:40.834570+0000","last_unstale":"2024-09-11T08:00:46.143324+0000","last_undegraded":"2024-09-11T08:00:46.143324+0000","last_fullsized":"2024-09-11T08:00:46.143324+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:27:26.508977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.142416+0000","last_change":"2024-09-11T08:00:43.067972+0000","last_active":"2024-09-11T08:00:46.142416+0000","last_peered":"2024-09-11T08:00:46.142416+0000","last_clean":"2024-09-11T08:00:46.142416+0000","last_became_active":"2024-09-11T08:00:43.067705+0000","last_became_peered":"2024-09-11T08:00:43.067705+0000","last_unstale":"2024-09-11T08:00:46.142416+0000","last_undegraded":"2024-09-11T08:00:46.142416+0000","last_fullsized":"2024-09-11T08:00:46.142416+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:03:23.242396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.200427+0000","last_change":"2024-09-11T08:00:36.798078+0000","last_active":"2024-09-11T08:00:46.200427+0000","last_peered":"2024-09-11T08:00:46.200427+0000","last_clean":"2024-09-11T08:00:46.200427+0000","last_became_active":"2024-09-11T08:00:36.797823+0000","last_became_peered":"2024-09-11T08:00:36.797823+0000","last_unstale":"2024-09-11T08:00:46.200427+0000","last_undegraded":"2024-09-11T08:00:46.200427+0000","last_fullsized":"2024-09-11T08:00:46.200427+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:04.709230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.227208+0000","last_change":"2024-09-11T08:00:38.818066+0000","last_active":"2024-09-11T08:00:46.227208+0000","last_peered":"2024-09-11T08:00:46.227208+0000","last_clean":"2024-09-11T08:00:46.227208+0000","last_became_active":"2024-09-11T08:00:38.817773+0000","last_became_peered":"2024-09-11T08:00:38.817773+0000","last_unstale":"2024-09-11T08:00:46.227208+0000","last_undegraded":"2024-09-11T08:00:46.227208+0000","last_fullsized":"2024-09-11T08:00:46.227208+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:18:09.442656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.144321+0000","last_change":"2024-09-11T08:00:40.835851+0000","last_active":"2024-09-11T08:00:46.144321+0000","last_peered":"2024-09-11T08:00:46.144321+0000","last_clean":"2024-09-11T08:00:46.144321+0000","last_became_active":"2024-09-11T08:00:40.835507+0000","last_became_peered":"2024-09-11T08:00:40.835507+0000","last_unstale":"2024-09-11T08:00:46.144321+0000","last_undegraded":"2024-09-11T08:00:46.144321+0000","last_fullsized":"2024-09-11T08:00:46.144321+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:28:25.572828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222745+0000","last_change":"2024-09-11T08:00:43.069092+0000","last_active":"2024-09-11T08:00:46.222745+0000","last_peered":"2024-09-11T08:00:46.222745+0000","last_clean":"2024-09-11T08:00:46.222745+0000","last_became_active":"2024-09-11T08:00:43.067854+0000","last_became_peered":"2024-09-11T08:00:43.067854+0000","last_unstale":"2024-09-11T08:00:46.222745+0000","last_undegraded":"2024-09-11T08:00:46.222745+0000","last_fullsized":"2024-09-11T08:00:46.222745+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:41.816636+0000","last_clean_scrub_stamp":"2024-09-11T08:00:41.816636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:19.226214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222401+0000","last_change":"2024-09-11T08:00:36.805092+0000","last_active":"2024-09-11T08:00:46.222401+0000","last_peered":"2024-09-11T08:00:46.222401+0000","last_clean":"2024-09-11T08:00:46.222401+0000","last_became_active":"2024-09-11T08:00:36.804684+0000","last_became_peered":"2024-09-11T08:00:36.804684+0000","last_unstale":"2024-09-11T08:00:46.222401+0000","last_undegraded":"2024-09-11T08:00:46.222401+0000","last_fullsized":"2024-09-11T08:00:46.222401+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:35.784345+0000","last_clean_scrub_stamp":"2024-09-11T08:00:35.784345+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:38:28.554646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.222474+0000","last_change":"2024-09-11T08:00:38.819139+0000","last_active":"2024-09-11T08:00:46.222474+0000","last_peered":"2024-09-11T08:00:46.222474+0000","last_clean":"2024-09-11T08:00:46.222474+0000","last_became_active":"2024-09-11T08:00:38.818876+0000","last_became_peered":"2024-09-11T08:00:38.818876+0000","last_unstale":"2024-09-11T08:00:46.222474+0000","last_undegraded":"2024-09-11T08:00:46.222474+0000","last_fullsized":"2024-09-11T08:00:46.222474+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:37.793733+0000","last_clean_scrub_stamp":"2024-09-11T08:00:37.793733+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:42:19.097088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T08:00:46.142060+0000","last_change":"2024-09-11T08:00:40.827187+0000","last_active":"2024-09-11T08:00:46.142060+0000","last_peered":"2024-09-11T08:00:46.142060+0000","last_clean":"2024-09-11T08:00:46.142060+0000","last_became_active":"2024-09-11T08:00:40.826414+0000","last_became_peered":"2024-09-11T08:00:40.826414+0000","last_unstale":"2024-09-11T08:00:46.142060+0000","last_undegraded":"2024-09-11T08:00:46.142060+0000","last_fullsized":"2024-09-11T08:00:46.142060+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T08:00:39.809627+0000","last_clean_scrub_stamp":"2024-09-11T08:00:39.809627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:34:01.256624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430229,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298536,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446616,"statfs":{"total":95995035648,"available":95689334784,"internally_reserved":0,"allocated":1212416,"data_stored":823517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.39100000000000001}]},{"osd":1,"last update":"Wed Sep 11 08:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.41199999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.40200000000000002}]},{"osd":3,"last update":"Wed Sep 11 08:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.432},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.42199999999999999}]},{"osd":4,"last update":"Wed Sep 11 08:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.36499999999999999}]},{"osd":5,"last update":"Wed Sep 11 08:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.38},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.33600000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.34699999999999998}]}]},{"osd":6,"up_from":42,"seq":180388626457,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298516,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446636,"statfs":{"total":95995035648,"available":95689355264,"internally_reserved":0,"allocated":1191936,"data_stored":822916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:01:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.443}]},{"osd":1,"last update":"Wed Sep 11 08:01:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.499},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.36899999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:01:00 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.48499999999999999}]},{"osd":3,"last update":"Wed Sep 11 08:01:00 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.432},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.54100000000000004}]},{"osd":4,"last update":"Wed Sep 11 08:01:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.38200000000000001}]},{"osd":5,"last update":"Wed Sep 11 08:01:00 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.52600000000000002}]},{"osd":7,"last update":"Wed Sep 11 08:01:27 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.51100000000000001}]}]},{"osd":5,"up_from":36,"seq":154618822688,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297932,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447220,"statfs":{"total":95995035648,"available":95689953280,"internally_reserved":0,"allocated":589824,"data_stored":231366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:01:42 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52400000000000002,"15min":0.52700000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.66000000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.499,"15min":0.503},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66600000000000004,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.54000000000000004}]},{"osd":1,"last update":"Wed Sep 11 08:01:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.36299999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.66600000000000004,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.54900000000000004}]},{"osd":2,"last update":"Wed Sep 11 08:01:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.50900000000000001,"15min":0.50800000000000001},"min":{"1min":0.35399999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.51600000000000001,"15min":0.51900000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.57199999999999995}]},{"osd":3,"last update":"Wed Sep 11 08:01:42 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.49299999999999999,"15min":0.496},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.624,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51500000000000001,"15min":0.51300000000000001},"min":{"1min":0.38100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.60399999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:01:42 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38700000000000001,"15min":0.38800000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.58199999999999996,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.39800000000000002,"15min":0.39200000000000002},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.46200000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:01:04 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.58499999999999996}]},{"osd":7,"last update":"Wed Sep 11 08:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.48999999999999999}]}]},{"osd":4,"up_from":30,"seq":128849018916,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297956,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447196,"statfs":{"total":95995035648,"available":95689928704,"internally_reserved":0,"allocated":614400,"data_stored":232113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.499,"15min":0.497},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64700000000000002,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.47599999999999998,"15min":0.47299999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.45800000000000002}]},{"osd":1,"last update":"Wed Sep 11 08:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.76000000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.503},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.48099999999999998,"15min":0.47599999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.44800000000000001}]},{"osd":2,"last update":"Wed Sep 11 08:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.51900000000000002,"15min":0.51600000000000001},"min":{"1min":0.36099999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.47599999999999998}]},{"osd":3,"last update":"Wed Sep 11 08:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.503,"15min":0.499},"min":{"1min":0.41799999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67200000000000004,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.50900000000000001,"15min":0.50700000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.46500000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:01:39 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.36299999999999999,"15min":0.35299999999999998},"min":{"1min":0.27800000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.39800000000000002,"15min":0.39000000000000001},"min":{"1min":0.312,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.33800000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:01:00 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"last":0.307}]},{"osd":7,"last update":"Wed Sep 11 08:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.36199999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182440,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297972,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447180,"statfs":{"total":95995035648,"available":95689912320,"internally_reserved":0,"allocated":630784,"data_stored":232464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36499999999999999,"15min":0.36399999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.219},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.26800000000000002,"5min":0.22,"15min":0.22},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.33000000000000002}]},{"osd":1,"last update":"Wed Sep 11 08:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.41499999999999998,"15min":0.41599999999999998},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.313},{"interface":"front","average":{"1min":0.371,"5min":0.39600000000000002,"15min":0.40000000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.49099999999999999,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.29899999999999999}]},{"osd":2,"last update":"Wed Sep 11 08:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.42299999999999999,"15min":0.42699999999999999},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.53400000000000003,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.25},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.441,"15min":0.44400000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.28399999999999997}]},{"osd":4,"last update":"Wed Sep 11 08:01:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.35999999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.73899999999999999,"5min":0.753,"15min":0.753},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.53700000000000003,"15min":0.54000000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71599999999999997,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.83399999999999996}]},{"osd":5,"last update":"Wed Sep 11 08:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.378,"5min":0.314,"15min":0.314},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56100000000000005,"15min":0.56799999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73499999999999999,"5min":0.751,"15min":0.751},"last":0.72899999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:01:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.58899999999999997}]},{"osd":7,"last update":"Wed Sep 11 08:01:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.68999999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297940,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447212,"statfs":{"total":95995035648,"available":95689945088,"internally_reserved":0,"allocated":598016,"data_stored":233095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.35499999999999998,"15min":0.36199999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.5,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.34899999999999998,"15min":0.34799999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.49399999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.27100000000000002}]},{"osd":1,"last update":"Wed Sep 11 08:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.36299999999999999,"15min":0.36599999999999999},"min":{"1min":0.28999999999999998,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.443,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47099999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.248}]},{"osd":3,"last update":"Wed Sep 11 08:01:59 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.40899999999999997,"15min":0.41599999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.436,"15min":0.442},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.62,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.373}]},{"osd":4,"last update":"Wed Sep 11 08:01:16 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52500000000000002,"15min":0.52700000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.54000000000000004,"15min":0.53700000000000003},"min":{"1min":0.42999999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.46899999999999997}]},{"osd":5,"last update":"Wed Sep 11 08:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53700000000000003,"15min":0.53500000000000003},"min":{"1min":0.39300000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.41899999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:00:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.495}]},{"osd":7,"last update":"Wed Sep 11 08:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.432}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297996,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447156,"statfs":{"total":95995035648,"available":95689887744,"internally_reserved":0,"allocated":655360,"data_stored":233649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 08:01:11 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.33600000000000002,"15min":0.32000000000000001},"min":{"1min":0.217,"5min":0.19,"15min":0.19},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.32800000000000001},"min":{"1min":0.24399999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.46000000000000002,"5min":0.504,"15min":0.504},"last":0.377}]},{"osd":2,"last update":"Wed Sep 11 08:01:35 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.38200000000000001,"15min":0.376},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.52000000000000002,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.39700000000000002,"15min":0.39100000000000001},"min":{"1min":0.30099999999999999,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.54600000000000004,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.34300000000000003}]},{"osd":3,"last update":"Wed Sep 11 08:00:58 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.39000000000000001,"15min":0.39200000000000002},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.55300000000000005,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.40799999999999997,"15min":0.40999999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.35599999999999998}]},{"osd":4,"last update":"Wed Sep 11 08:01:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56299999999999994,"15min":0.56599999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83899999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.496}]},{"osd":5,"last update":"Wed Sep 11 08:01:40 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.42399999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.78200000000000003,"5min":0.875,"15min":0.875},"last":0.46100000000000002}]},{"osd":6,"last update":"Wed Sep 11 08:01:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.50600000000000001}]},{"osd":7,"last update":"Wed Sep 11 08:01:25 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.53600000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298532,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446620,"statfs":{"total":95995035648,"available":95689338880,"internally_reserved":0,"allocated":1208320,"data_stored":822387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 08:01:11 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.31900000000000001,"15min":0.30599999999999999},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.33700000000000002,"15min":0.32100000000000001},"min":{"1min":0.24099999999999999,"5min":0.192,"15min":0.192},"max":{"1min":0.52000000000000002,"5min":0.56599999999999995,"15min":0.56599999999999995},"last":0.35499999999999998}]},{"osd":2,"last update":"Wed Sep 11 08:01:32 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.33300000000000002,"5min":0.35399999999999998,"15min":0.35799999999999998},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.34300000000000003}]},{"osd":3,"last update":"Wed Sep 11 08:01:57 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.41699999999999998,"15min":0.42699999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.502,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.41399999999999998,"15min":0.42999999999999999},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.497,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.441}]},{"osd":4,"last update":"Wed Sep 11 08:01:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.39100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73299999999999998,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56100000000000005,"15min":0.56299999999999994},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.73499999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.52700000000000002}]},{"osd":5,"last update":"Wed Sep 11 08:01:38 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.373,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.374},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.48299999999999998}]},{"osd":6,"last update":"Wed Sep 11 08:01:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.61699999999999999}]},{"osd":7,"last update":"Wed Sep 11 08:01:24 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.56799999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T08:02:02.827 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T08:02:02.827 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T08:02:02.827 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-11T08:02:02.827 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid 2051892c-7013-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-11T08:02:02.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:02 smithi146 ceph-mon[33158]: from='client.24655 v1:172.21.15.102:0/3705670856' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T08:02:03.027 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:02 smithi102 podman[77557]: 2024-09-11 08:02:02.67029584 +0000 UTC m=+3.041401857 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-11T08:02:03.027 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:02 smithi102 podman[77557]: 2024-09-11 08:02:02.919918191 +0000 UTC m=+3.291024208 container create 6f44a611aab2970264a5b67d2745838a8864cc21baa79d9e1d0102e1bf8c3109 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:02:03.266 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T08:02:03.392 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 podman[77557]: 2024-09-11 08:02:03.130579664 +0000 UTC m=+3.501685692 container init 6f44a611aab2970264a5b67d2745838a8864cc21baa79d9e1d0102e1bf8c3109 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:02:03.392 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 podman[77557]: 2024-09-11 08:02:03.135689498 +0000 UTC m=+3.506795525 container start 6f44a611aab2970264a5b67d2745838a8864cc21baa79d9e1d0102e1bf8c3109 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:02:03.392 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.139Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-11T08:02:03.392 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.139Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-11T08:02:03.392 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.139Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-11T08:02:03.392 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-11T08:02:03.393 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-11T08:02:03.393 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-11T08:02:03.393 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-11T08:02:03.393 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=arp 2024-09-11T08:02:03.393 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-11T08:02:03.393 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-11T08:02:03.393 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-11T08:02:03.393 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-11T08:02:03.393 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-11T08:02:03.393 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-11T08:02:03.394 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-11T08:02:03.394 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-11T08:02:03.394 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=edac 2024-09-11T08:02:03.394 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-11T08:02:03.394 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-11T08:02:03.394 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-11T08:02:03.394 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-11T08:02:03.394 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-11T08:02:03.394 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-11T08:02:03.394 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-11T08:02:03.394 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-11T08:02:03.395 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-11T08:02:03.395 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-11T08:02:03.395 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-11T08:02:03.395 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-11T08:02:03.395 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-11T08:02:03.395 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-11T08:02:03.395 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-11T08:02:03.395 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-11T08:02:03.395 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=os 2024-09-11T08:02:03.395 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-11T08:02:03.395 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-11T08:02:03.395 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-11T08:02:03.395 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-11T08:02:03.395 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-11T08:02:03.396 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-11T08:02:03.396 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-11T08:02:03.396 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=stat 2024-09-11T08:02:03.396 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-11T08:02:03.396 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-11T08:02:03.396 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.140Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-11T08:02:03.396 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.141Z caller=node_exporter.go:117 level=info collector=time 2024-09-11T08:02:03.396 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.141Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-11T08:02:03.396 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.141Z caller=node_exporter.go:117 level=info collector=uname 2024-09-11T08:02:03.396 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.141Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-11T08:02:03.396 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.141Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-11T08:02:03.396 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.141Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-11T08:02:03.396 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.141Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-11T08:02:03.397 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a[77772]: ts=2024-09-11T08:02:03.141Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-11T08:02:03.397 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 bash[77557]: 6f44a611aab2970264a5b67d2745838a8864cc21baa79d9e1d0102e1bf8c3109 2024-09-11T08:02:03.397 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:02:03 smithi102 systemd[1]: Started Ceph node-exporter.a for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T08:02:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-mon[41371]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:03.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-mon[29849]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:03.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:03.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:03.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:03 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:03.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:03 smithi146 ceph-mon[33158]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:03.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:03 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:03.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:03 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:03.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:03 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:04.434 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:04 smithi146 bash[66915]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-11T08:02:04.718 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:04 smithi102 ceph-mon[29849]: Deploying daemon node-exporter.b on smithi146 2024-09-11T08:02:04.718 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:04 smithi102 ceph-mon[41371]: Deploying daemon node-exporter.b on smithi146 2024-09-11T08:02:04.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.908+0000 7f716eae0640 1 -- 172.21.15.102:0/4099760897 >> v1:172.21.15.102:6790/0 conn(0x7f71681042a0 legacy=0x7f7168106680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:02:04.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.908+0000 7f716eae0640 1 -- 172.21.15.102:0/4099760897 shutdown_connections 2024-09-11T08:02:04.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.908+0000 7f716eae0640 1 -- 172.21.15.102:0/4099760897 >> 172.21.15.102:0/4099760897 conn(0x7f71680752c0 msgr2=0x7f71680f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:02:04.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.908+0000 7f716eae0640 1 -- 172.21.15.102:0/4099760897 shutdown_connections 2024-09-11T08:02:04.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.908+0000 7f716eae0640 1 -- 172.21.15.102:0/4099760897 wait complete. 2024-09-11T08:02:04.933 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.908+0000 7f716eae0640 1 Processor -- start 2024-09-11T08:02:04.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.908+0000 7f716eae0640 1 -- start start 2024-09-11T08:02:04.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f716eae0640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f716811e240 con 0x7f7168101530 2024-09-11T08:02:04.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f716eae0640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f716811f400 con 0x7f7168107e00 2024-09-11T08:02:04.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f716eae0640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7168120600 con 0x7f71681042a0 2024-09-11T08:02:04.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f716c87c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f7168101530 0x7f7168065010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:55212/0 (socket says 172.21.15.102:55212) 2024-09-11T08:02:04.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f716c87c640 1 -- 172.21.15.102:0/30189104 learned_addr learned my addr 172.21.15.102:0/30189104 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:02:04.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f715ffff640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7f71681042a0 0x7f7168065720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:51570/0 (socket says 172.21.15.102:51570) 2024-09-11T08:02:04.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1337130920 0 0) 0x7f7168120600 con 0x7f71681042a0 2024-09-11T08:02:04.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7150003120 con 0x7f71681042a0 2024-09-11T08:02:04.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1988751328 0 0) 0x7f716811e240 con 0x7f7168101530 2024-09-11T08:02:04.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7168120600 con 0x7f7168101530 2024-09-11T08:02:04.934 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2880355742 0 0) 0x7f716811f400 con 0x7f7168107e00 2024-09-11T08:02:04.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f716811e240 con 0x7f7168107e00 2024-09-11T08:02:04.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2210771579 0 0) 0x7f7150003120 con 0x7f71681042a0 2024-09-11T08:02:04.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f716811f400 con 0x7f71681042a0 2024-09-11T08:02:04.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f7158001820 con 0x7f71681042a0 2024-09-11T08:02:04.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2427647369 0 0) 0x7f7168120600 con 0x7f7168101530 2024-09-11T08:02:04.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7150003120 con 0x7f7168101530 2024-09-11T08:02:04.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 488692033 0 0) 0x7f716811e240 con 0x7f7168107e00 2024-09-11T08:02:04.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7168120600 con 0x7f7168107e00 2024-09-11T08:02:04.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f71640019f0 con 0x7f7168101530 2024-09-11T08:02:04.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f7160001bc0 con 0x7f7168107e00 2024-09-11T08:02:04.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 551812295 0 0) 0x7f716811f400 con 0x7f71681042a0 2024-09-11T08:02:04.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 >> v1:172.21.15.146:6789/0 conn(0x7f7168107e00 legacy=0x7f716811c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:02:04.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 >> v1:172.21.15.102:6789/0 conn(0x7f7168101530 legacy=0x7f7168065010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:02:04.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.909+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7168121800 con 0x7f71681042a0 2024-09-11T08:02:04.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.910+0000 7f716eae0640 1 -- 172.21.15.102:0/30189104 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71681207f0 con 0x7f71681042a0 2024-09-11T08:02:04.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.910+0000 7f716eae0640 1 -- 172.21.15.102:0/30189104 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7168120c40 con 0x7f71681042a0 2024-09-11T08:02:04.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.910+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f7158001600 con 0x7f71681042a0 2024-09-11T08:02:04.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.910+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f71580034e0 con 0x7f71681042a0 2024-09-11T08:02:04.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.911+0000 7f716eae0640 1 -- 172.21.15.102:0/30189104 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7154005010 con 0x7f71681042a0 2024-09-11T08:02:04.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.911+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7f7158002180 con 0x7f71681042a0 2024-09-11T08:02:04.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.912+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 376332686 0 0) 0x7f7158089e50 con 0x7f71681042a0 2024-09-11T08:02:04.936 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:04.916+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f7158056b40 con 0x7f71681042a0 2024-09-11T08:02:04.937 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:04 smithi146 ceph-mon[33158]: Deploying daemon node-exporter.b on smithi146 2024-09-11T08:02:04.937 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:02:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:02:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:02:05.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:05.107+0000 7f716eae0640 1 -- 172.21.15.102:0/30189104 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7154005840 con 0x7f71681042a0 2024-09-11T08:02:05.108 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:05.107+0000 7f715d7fa640 1 -- 172.21.15.102:0/30189104 <== mon.2 v1:172.21.15.102:6790/0 10 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (unknown 3730786198 0 4185958460) 0x7f71580564e0 con 0x7f71681042a0 2024-09-11T08:02:05.108 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-11T08:02:05.109 INFO:teuthology.orchestra.run.smithi102.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T08:02:05.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:05.109+0000 7f716eae0640 1 -- 172.21.15.102:0/30189104 >> v1:172.21.15.102:6800/4281557748 conn(0x7f7150070040 legacy=0x7f71500724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:02:05.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:05.109+0000 7f716eae0640 1 -- 172.21.15.102:0/30189104 >> v1:172.21.15.102:6790/0 conn(0x7f71681042a0 legacy=0x7f7168065720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:02:05.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:05.109+0000 7f716eae0640 1 -- 172.21.15.102:0/30189104 shutdown_connections 2024-09-11T08:02:05.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:05.109+0000 7f716eae0640 1 -- 172.21.15.102:0/30189104 >> 172.21.15.102:0/30189104 conn(0x7f71680752c0 msgr2=0x7f716810aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:02:05.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:05.109+0000 7f716eae0640 1 -- 172.21.15.102:0/30189104 shutdown_connections 2024-09-11T08:02:05.111 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-11T08:02:05.109+0000 7f716eae0640 1 -- 172.21.15.102:0/30189104 wait complete. 2024-09-11T08:02:05.434 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:05 smithi146 bash[66915]: Getting image source signatures 2024-09-11T08:02:05.434 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:05 smithi146 bash[66915]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-11T08:02:05.434 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:05 smithi146 bash[66915]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-11T08:02:05.434 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:05 smithi146 bash[66915]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-11T08:02:05.700 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:05 smithi146 ceph-mon[33158]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:05.700 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:05 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/30189104' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T08:02:05.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:05 smithi102 ceph-mon[29849]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:05.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:05 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/30189104' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T08:02:05.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:05 smithi102 ceph-mon[41371]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:05.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:05 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/30189104' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T08:02:06.042 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-11T08:02:06.042 INFO:tasks.cephadm:Setup complete, yielding 2024-09-11T08:02:06.042 INFO:teuthology.run_tasks:Running task workunit... 2024-09-11T08:02:06.058 INFO:tasks.workunit:Pulling workunits from ref d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:02:06.058 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-11T08:02:06.058 DEBUG:teuthology.orchestra.run.smithi102:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-11T08:02:06.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:02:06.075 INFO:teuthology.orchestra.run.smithi102.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-11T08:02:06.076 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-11T08:02:06.132 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-11T08:02:06.132 DEBUG:teuthology.orchestra.run.smithi102:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-11T08:02:06.188 INFO:tasks.workunit:timeout=3h 2024-09-11T08:02:06.189 INFO:tasks.workunit:cleanup=True 2024-09-11T08:02:06.189 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T08:02:06.245 INFO:tasks.workunit.client.0.smithi102.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-11T08:02:07.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:07 smithi102 ceph-mon[41371]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:07.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:07 smithi102 ceph-mon[29849]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:07.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:07 smithi146 ceph-mon[33158]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:09.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:09 smithi146 ceph-mon[33158]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:09.684 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:02:09 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:02:09.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:09 smithi102 ceph-mon[41371]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:09.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:09 smithi102 ceph-mon[29849]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:10.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:10 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:02:10.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:10 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:02:10.919 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:10 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:02:10.920 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:10 smithi146 bash[66915]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-11T08:02:11.183 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:10 smithi146 bash[66915]: Writing manifest to image destination 2024-09-11T08:02:11.542 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 podman[66915]: 2024-09-11 08:02:11.19150036 +0000 UTC m=+7.072646070 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-11T08:02:11.542 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 podman[66915]: 2024-09-11 08:02:11.289964623 +0000 UTC m=+7.171110319 container create e4a63719b0a9b4b8dd964943bd319e32438794cfc50449c4e82b78945dc34257 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:02:11.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:11 smithi102 ceph-mon[29849]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:11.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:11 smithi102 ceph-mon[41371]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:11.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-mon[33158]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:11.934 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 podman[66915]: 2024-09-11 08:02:11.541649831 +0000 UTC m=+7.422795527 container init e4a63719b0a9b4b8dd964943bd319e32438794cfc50449c4e82b78945dc34257 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:02:11.934 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 podman[66915]: 2024-09-11 08:02:11.548455201 +0000 UTC m=+7.429600908 container start e4a63719b0a9b4b8dd964943bd319e32438794cfc50449c4e82b78945dc34257 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:02:11.934 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.552Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-11T08:02:11.934 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.552Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-11T08:02:11.934 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-11T08:02:11.935 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-11T08:02:11.935 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-11T08:02:11.935 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-11T08:02:11.935 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-11T08:02:11.935 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=arp 2024-09-11T08:02:11.935 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-11T08:02:11.935 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-11T08:02:11.935 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-11T08:02:11.935 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-11T08:02:11.935 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-11T08:02:11.935 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-11T08:02:11.936 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-11T08:02:11.936 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-11T08:02:11.936 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=edac 2024-09-11T08:02:11.936 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-11T08:02:11.936 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-11T08:02:11.936 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-11T08:02:11.936 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-11T08:02:11.936 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-11T08:02:11.937 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-11T08:02:11.937 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-11T08:02:11.937 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-11T08:02:11.937 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-11T08:02:11.937 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-11T08:02:11.937 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-11T08:02:11.937 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-11T08:02:11.937 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-11T08:02:11.937 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-11T08:02:11.937 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-11T08:02:11.937 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-11T08:02:11.937 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=os 2024-09-11T08:02:11.938 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-11T08:02:11.938 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-11T08:02:11.938 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-11T08:02:11.938 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-11T08:02:11.938 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-11T08:02:11.938 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-11T08:02:11.938 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-11T08:02:11.938 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=stat 2024-09-11T08:02:11.938 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-11T08:02:11.938 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-11T08:02:11.938 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-11T08:02:11.938 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=time 2024-09-11T08:02:11.938 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-11T08:02:11.939 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=uname 2024-09-11T08:02:11.939 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-11T08:02:11.939 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-11T08:02:11.939 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.553Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-11T08:02:11.939 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.554Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-11T08:02:11.939 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b[67001]: ts=2024-09-11T08:02:11.554Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-11T08:02:11.939 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 bash[66915]: e4a63719b0a9b4b8dd964943bd319e32438794cfc50449c4e82b78945dc34257 2024-09-11T08:02:11.939 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:02:11 smithi146 systemd[1]: Started Ceph node-exporter.b for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T08:02:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:12 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:12 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:12 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:12 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:12 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:12 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:02:13.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:12 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:13.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:12 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:13.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:12 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:13.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:12 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:13.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:12 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:13.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:12 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:02:13.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:12 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:13.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:12 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:13.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:12 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:13.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:12 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:13.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:12 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:13.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:12 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:02:14.084 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:13 smithi146 ceph-mon[33158]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:14.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:13 smithi102 ceph-mon[29849]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:14.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:13 smithi102 ceph-mon[41371]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:14.905 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:02:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:02:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:02:14.905 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:14 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2024-09-11T08:02:14.905 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:14 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-09-11T08:02:14.905 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:14 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2024-09-11T08:02:14.906 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:14 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2024-09-11T08:02:14.906 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:14 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-09-11T08:02:14.906 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:14 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-09-11T08:02:14.906 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:14 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2024-09-11T08:02:14.906 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:14 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-09-11T08:02:14.906 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:14 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:02:14.906 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:14 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:14.906 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:14 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:14.906 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:14 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:15.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2024-09-11T08:02:15.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-09-11T08:02:15.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2024-09-11T08:02:15.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2024-09-11T08:02:15.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-09-11T08:02:15.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-09-11T08:02:15.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2024-09-11T08:02:15.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-09-11T08:02:15.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:02:15.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:15.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:15.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:15.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2024-09-11T08:02:15.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-09-11T08:02:15.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2024-09-11T08:02:15.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2024-09-11T08:02:15.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-09-11T08:02:15.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-09-11T08:02:15.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2024-09-11T08:02:15.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-09-11T08:02:15.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:02:15.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:15.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:15.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:14 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:15.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:15 smithi146 ceph-mon[33158]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:15.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:15 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]': finished 2024-09-11T08:02:15.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:15 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-09-11T08:02:15.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:15 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]': finished 2024-09-11T08:02:15.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:15 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-11T08:02:15.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:15 smithi146 ceph-mon[33158]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T08:02:16.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:15 smithi102 ceph-mon[29849]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:16.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:15 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]': finished 2024-09-11T08:02:16.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:15 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-09-11T08:02:16.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:15 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]': finished 2024-09-11T08:02:16.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:15 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-11T08:02:16.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:15 smithi102 ceph-mon[29849]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T08:02:16.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:15 smithi102 ceph-mon[41371]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:16.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:15 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]': finished 2024-09-11T08:02:16.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:15 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-09-11T08:02:16.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:15 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]': finished 2024-09-11T08:02:16.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:15 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-09-11T08:02:16.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:15 smithi102 ceph-mon[41371]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T08:02:16.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:16 smithi102 ceph-mon[29849]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T08:02:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:16 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:16 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:16 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:02:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:16 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:02:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:16 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:16 smithi102 ceph-mon[41371]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T08:02:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:16 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:16 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:16 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:02:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:16 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:02:16.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:16 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:17.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:16 smithi146 ceph-mon[33158]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T08:02:17.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:16 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:17.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:16 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:17.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:16 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:02:17.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:16 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:02:17.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:16 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:17.879 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:17 smithi102 ceph-mon[41371]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T08:02:17.879 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:17 smithi102 ceph-mon[41371]: Reconfiguring daemon alertmanager.a on smithi102 2024-09-11T08:02:17.879 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:17 smithi102 ceph-mon[41371]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:02:17.879 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:17 smithi102 ceph-mon[41371]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-11T08:02:17.880 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:17 smithi102 ceph-mon[29849]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T08:02:17.880 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:17 smithi102 ceph-mon[29849]: Reconfiguring daemon alertmanager.a on smithi102 2024-09-11T08:02:17.880 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:17 smithi102 ceph-mon[29849]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:02:17.880 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:17 smithi102 ceph-mon[29849]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-11T08:02:18.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:17 smithi146 ceph-mon[33158]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T08:02:18.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:17 smithi146 ceph-mon[33158]: Reconfiguring daemon alertmanager.a on smithi102 2024-09-11T08:02:18.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:17 smithi146 ceph-mon[33158]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:02:18.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:17 smithi146 ceph-mon[33158]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-11T08:02:19.646 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:19 smithi102 systemd[1]: Stopping Ceph alertmanager.a for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:02:19.684 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:02:19 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:02:20.035 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:19 smithi102 ceph-mon[41371]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:02:20.035 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:19 smithi102 ceph-mon[29849]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:02:20.035 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[76080]: ts=2024-09-11T08:02:19.743Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:02:20.035 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:19 smithi102 podman[78834]: 2024-09-11 08:02:19.764638963 +0000 UTC m=+0.116756521 container died afe38f922c4b48b21bfe4450e84cf647c6d653b066ab106dde766522c623c0a6 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:02:20.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:19 smithi146 ceph-mon[33158]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:02:20.395 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:20 smithi102 podman[78834]: 2024-09-11 08:02:20.141235011 +0000 UTC m=+0.493352569 container cleanup afe38f922c4b48b21bfe4450e84cf647c6d653b066ab106dde766522c623c0a6 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:02:20.395 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:20 smithi102 bash[78834]: ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a 2024-09-11T08:02:20.697 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:20 smithi102 podman[78851]: 2024-09-11 08:02:20.400713076 +0000 UTC m=+0.636938997 container remove afe38f922c4b48b21bfe4450e84cf647c6d653b066ab106dde766522c623c0a6 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:02:20.697 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:20 smithi102 podman[78851]: 2024-09-11 08:02:20.493193561 +0000 UTC m=+0.729419483 volume remove faf3a9f569c358e0c7e27efb37dbeb58e1bfa39359c714adfe3ca8ca529c9ee7 2024-09-11T08:02:21.028 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:20 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:02:21.029 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:20 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:02:21.029 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:20 smithi102 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-11T08:02:21.029 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:20 smithi102 systemd[1]: Stopped Ceph alertmanager.a for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T08:02:21.029 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:20 smithi102 systemd[1]: Starting Ceph alertmanager.a for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:02:21.029 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:20 smithi102 podman[78945]: 2024-09-11 08:02:20.822527882 +0000 UTC m=+0.016193970 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-11T08:02:21.029 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:20 smithi102 podman[78945]: 2024-09-11 08:02:20.944582641 +0000 UTC m=+0.138248760 volume create a46821dc73bc5431bd08c28065be24a775a6537b9b830e8281916cce7619b7fd 2024-09-11T08:02:21.029 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:21 smithi102 podman[78945]: 2024-09-11 08:02:21.028171267 +0000 UTC m=+0.221837339 container create 5068c22cecbc535e57a2e446c714409f1d0fc43c453113db057a942ad6a16810 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:02:21.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:20 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:02:21.646 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:21 smithi102 podman[78945]: 2024-09-11 08:02:21.289319784 +0000 UTC m=+0.482985860 container init 5068c22cecbc535e57a2e446c714409f1d0fc43c453113db057a942ad6a16810 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:02:21.646 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:21 smithi102 podman[78945]: 2024-09-11 08:02:21.294427682 +0000 UTC m=+0.488093756 container start 5068c22cecbc535e57a2e446c714409f1d0fc43c453113db057a942ad6a16810 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:02:21.646 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:21 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:02:21.305Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-11T08:02:21.646 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:21 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:02:21.305Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-11T08:02:21.646 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:21 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:02:21.307Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.102 port=9094 2024-09-11T08:02:21.646 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:21 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:02:21.308Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-11T08:02:21.646 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:21 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:02:21.341Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T08:02:21.646 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:21 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:02:21.342Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T08:02:21.646 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:21 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:02:21.345Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-11T08:02:21.646 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:21 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:02:21.345Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-11T08:02:21.646 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:21 smithi102 bash[78945]: 5068c22cecbc535e57a2e446c714409f1d0fc43c453113db057a942ad6a16810 2024-09-11T08:02:21.647 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:21 smithi102 systemd[1]: Started Ceph alertmanager.a for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T08:02:22.100 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:21 smithi146 ceph-mon[33158]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-09-11T08:02:22.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:21 smithi146 ceph-mon[33158]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-11T08:02:22.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:21 smithi146 ceph-mon[33158]: Cluster is now healthy 2024-09-11T08:02:22.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:21 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:22.101 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:21 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:22.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:21 smithi102 ceph-mon[41371]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-09-11T08:02:22.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:21 smithi102 ceph-mon[41371]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-11T08:02:22.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:21 smithi102 ceph-mon[41371]: Cluster is now healthy 2024-09-11T08:02:22.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:21 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:22.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:21 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:22.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:21 smithi102 ceph-mon[29849]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-09-11T08:02:22.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:21 smithi102 ceph-mon[29849]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-11T08:02:22.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:21 smithi102 ceph-mon[29849]: Cluster is now healthy 2024-09-11T08:02:22.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:21 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:22.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:21 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:23.076 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:22 smithi146 ceph-mon[33158]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T08:02:23.077 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:22 smithi146 ceph-mon[33158]: Reconfiguring daemon prometheus.a on smithi146 2024-09-11T08:02:23.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:22 smithi102 ceph-mon[41371]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T08:02:23.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:22 smithi102 ceph-mon[41371]: Reconfiguring daemon prometheus.a on smithi146 2024-09-11T08:02:23.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:22 smithi102 ceph-mon[29849]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T08:02:23.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:22 smithi102 ceph-mon[29849]: Reconfiguring daemon prometheus.a on smithi146 2024-09-11T08:02:23.645 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:23 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:02:23.308Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000267534s 2024-09-11T08:02:24.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:23 smithi102 ceph-mon[29849]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-09-11T08:02:24.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:23 smithi102 ceph-mon[41371]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-09-11T08:02:24.159 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:23 smithi146 ceph-mon[33158]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 41 B/s, 1 objects/s recovering 2024-09-11T08:02:24.915 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:24 smithi146 systemd[1]: Stopping Ceph prometheus.a for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:02:24.915 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:02:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:02:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:02:25.184 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:02:24.914Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:02:25.184 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:02:24.915Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-11T08:02:25.184 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:02:24.915Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-11T08:02:25.184 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:02:24.915Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-11T08:02:25.184 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:02:24.915Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-11T08:02:25.184 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:02:24.915Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-11T08:02:25.184 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:02:24.915Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-11T08:02:25.185 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:02:24.915Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-11T08:02:25.185 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:02:24.915Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-11T08:02:25.185 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:02:24.915Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-11T08:02:25.185 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:02:24.915Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-11T08:02:25.185 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[64708]: ts=2024-09-11T08:02:24.916Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-11T08:02:25.185 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:24 smithi146 podman[67926]: 2024-09-11 08:02:24.925574036 +0000 UTC m=+0.118035798 container died bfcbdfbd6e87e5e3b100f011c0f9c34130198231252a0f18ae145a71d4dead1b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:02:25.580 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:25 smithi146 podman[67926]: 2024-09-11 08:02:25.329126933 +0000 UTC m=+0.521588686 container cleanup bfcbdfbd6e87e5e3b100f011c0f9c34130198231252a0f18ae145a71d4dead1b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:02:25.581 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:25 smithi146 bash[67926]: ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a 2024-09-11T08:02:25.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:25 smithi146 ceph-mon[33158]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 34 B/s, 1 objects/s recovering 2024-09-11T08:02:25.934 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:25 smithi146 podman[67943]: 2024-09-11 08:02:25.580458825 +0000 UTC m=+0.656008306 container remove bfcbdfbd6e87e5e3b100f011c0f9c34130198231252a0f18ae145a71d4dead1b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:02:25.934 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:25 smithi146 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-11T08:02:25.934 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:25 smithi146 systemd[1]: Stopped Ceph prometheus.a for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T08:02:25.934 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:25 smithi146 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@prometheus.a.service: Consumed 1.078s CPU time. 2024-09-11T08:02:25.934 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:25 smithi146 systemd[1]: Starting Ceph prometheus.a for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:02:26.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:25 smithi102 ceph-mon[41371]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 34 B/s, 1 objects/s recovering 2024-09-11T08:02:26.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:25 smithi102 ceph-mon[29849]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 34 B/s, 1 objects/s recovering 2024-09-11T08:02:26.229 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:25 smithi146 podman[68042]: 2024-09-11 08:02:25.863638685 +0000 UTC m=+0.017045404 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-11T08:02:26.229 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:25 smithi146 podman[68042]: 2024-09-11 08:02:25.972834026 +0000 UTC m=+0.126240734 container create 83beeb4391f4ce1f3c5137ce595b8cb93ab5910ba55c192dee2f7488f170346c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:02:26.229 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 podman[68042]: 2024-09-11 08:02:26.20017073 +0000 UTC m=+0.353577447 container init 83beeb4391f4ce1f3c5137ce595b8cb93ab5910ba55c192dee2f7488f170346c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:02:26.229 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 podman[68042]: 2024-09-11 08:02:26.204735837 +0000 UTC m=+0.358142546 container start 83beeb4391f4ce1f3c5137ce595b8cb93ab5910ba55c192dee2f7488f170346c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:02:26.645 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:02:26 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:02:26] ENGINE Bus STOPPING 2024-09-11T08:02:26.684 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:02:26.229Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-11T08:02:26.684 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:02:26.229Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-11T08:02:26.684 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:02:26.229Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi146 (none))" 2024-09-11T08:02:26.685 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:02:26.229Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-11T08:02:26.685 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:02:26.229Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-11T08:02:26.685 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:02:26.230Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-11T08:02:26.685 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:02:26.231Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-11T08:02:26.685 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:02:26.232Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-11T08:02:26.685 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:02:26.232Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-11T08:02:26.685 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:02:26.235Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-11T08:02:26.685 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:02:26.235Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.69µs 2024-09-11T08:02:26.685 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:02:26.235Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-11T08:02:26.685 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:02:26.235Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-11T08:02:26.685 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:02:26.235Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-11T08:02:26.686 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:02:26.235Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=22.149µs wal_replay_duration=559.871µs wbl_replay_duration=111ns total_replay_duration=599.939µs 2024-09-11T08:02:26.686 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:02:26.236Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-11T08:02:26.686 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:02:26.236Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-11T08:02:26.686 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:02:26.236Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-11T08:02:26.686 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:02:26.252Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.526676ms db_storage=752ns remote_storage=1.126µs web_handler=421ns query_engine=490ns scrape=382.471µs scrape_sd=31.908µs notify=26.556µs notify_sd=7.17µs rules=14.842554ms tracing=6.462µs 2024-09-11T08:02:26.686 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:02:26.252Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-11T08:02:26.686 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:02:26.252Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-11T08:02:26.686 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 bash[68042]: 83beeb4391f4ce1f3c5137ce595b8cb93ab5910ba55c192dee2f7488f170346c 2024-09-11T08:02:26.686 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:02:26 smithi146 systemd[1]: Started Ceph prometheus.a for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T08:02:27.026 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:02:26 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:02:26] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:02:27.026 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:02:26 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:02:26] ENGINE Bus STOPPED 2024-09-11T08:02:27.026 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:02:26 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:02:26] ENGINE Bus STARTING 2024-09-11T08:02:27.026 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:02:26 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:02:26] ENGINE Serving on http://:::9283 2024-09-11T08:02:27.026 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:02:26 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:02:26] ENGINE Bus STARTED 2024-09-11T08:02:27.026 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:02:26 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:02:26] ENGINE Bus STOPPING 2024-09-11T08:02:27.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:27.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:27.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:02:27.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:02:27.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi102.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:02:27.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi102.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:02:27.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:27.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:02:27.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:02:27.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi146.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:02:27.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi146.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:02:27.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:27.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[41371]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 96 B/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-09-11T08:02:27.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:02:27.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:02:27.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi146.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:02:27.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi146.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:02:27.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:27.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:02:27.648 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:02:27] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:02:27.648 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:02:27] ENGINE Bus STOPPED 2024-09-11T08:02:27.648 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:02:27] ENGINE Bus STARTING 2024-09-11T08:02:27.648 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:02:27] ENGINE Serving on http://:::9283 2024-09-11T08:02:27.648 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:02:27] ENGINE Bus STARTED 2024-09-11T08:02:27.648 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:02:27] ENGINE Bus STOPPING 2024-09-11T08:02:27.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:27.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:27.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:02:27.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:02:27.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi102.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:02:27.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi102.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:02:27.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:27.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:02:27.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:02:27.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi146.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:02:27.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi146.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:02:27.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:27.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[29849]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 96 B/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-09-11T08:02:27.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:02:27.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:02:27.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi146.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:02:27.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi146.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:02:27.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:27.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:02:27.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:27 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:27.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:27 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:27.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:27 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:02:27.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:27 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T08:02:27.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:27 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi102.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:02:27.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:27 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi102.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T08:02:27.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:27 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:27.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:27 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:02:27.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:27 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T08:02:27.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:27 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi146.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:02:27.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:27 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi146.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T08:02:27.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:27 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:27.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:27 smithi146 ceph-mon[33158]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 96 B/s wr, 3 op/s; 31 B/s, 0 objects/s recovering 2024-09-11T08:02:27.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:27 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:02:27.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:27 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T08:02:27.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:27 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi146.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:02:27.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:27 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi146.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T08:02:27.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:27 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:27.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:27 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:02:28.314 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:02:27] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:02:28.314 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:02:27] ENGINE Bus STOPPED 2024-09-11T08:02:28.314 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:02:27 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:02:27] ENGINE Bus STARTING 2024-09-11T08:02:28.314 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:02:28 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:02:28] ENGINE Serving on http://:::9283 2024-09-11T08:02:28.314 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:02:28 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:02:28] ENGINE Bus STARTED 2024-09-11T08:02:29.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:29 smithi146 ceph-mon[33158]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s; 27 B/s, 0 objects/s recovering 2024-09-11T08:02:29.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:29 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:02:29.684 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:02:29 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:02:29.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:29 smithi102 ceph-mon[41371]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s; 27 B/s, 0 objects/s recovering 2024-09-11T08:02:29.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:29 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:02:29.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:29 smithi102 ceph-mon[29849]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s; 27 B/s, 0 objects/s recovering 2024-09-11T08:02:29.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:29 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:02:30.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:30 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:02:30.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:30 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:02:30.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:30 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:02:31.514 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:02:31 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:02:31.311Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.00351431s 2024-09-11T08:02:31.515 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:31 smithi102 ceph-mon[29849]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 85 B/s wr, 3 op/s; 27 B/s, 0 objects/s recovering 2024-09-11T08:02:31.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:31 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:31.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:31 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:31.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:31 smithi102 ceph-mon[41371]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 85 B/s wr, 3 op/s; 27 B/s, 0 objects/s recovering 2024-09-11T08:02:31.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:31 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:31.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:31 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:31 smithi146 ceph-mon[33158]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 85 B/s wr, 3 op/s; 27 B/s, 0 objects/s recovering 2024-09-11T08:02:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:31 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:31 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:32.815 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:32 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:32.815 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:32 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:32.815 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:32 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:02:32.815 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:32 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:02:32.815 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:32 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:32.815 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:32 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:02:32.815 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:32 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:32.816 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:32 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:32.816 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:32 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:02:32.816 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:32 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:02:32.816 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:32 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:32.816 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:32 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:02:32.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:32 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:32.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:32 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:32.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:32 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:02:32.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:32 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:02:32.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:32 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:32.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:32 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:02:33.881 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:33 smithi102 ceph-mon[29849]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:33.881 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:33 smithi102 ceph-mon[41371]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:33.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:33 smithi146 ceph-mon[33158]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:34.859 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:02:34 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:02:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:02:35.804 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:35 smithi146 ceph-mon[33158]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:35.804 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:35 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:35.804 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:35 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:35.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:35 smithi102 ceph-mon[41371]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:35.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:35 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:35.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:35 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:35.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:35 smithi102 ceph-mon[29849]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:35.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:35 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:35.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:35 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:37.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:37 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:37.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:37 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:37.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:37 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:02:37.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:37 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:02:37.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:37 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:37.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:37 smithi102 ceph-mon[41371]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:37.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:37 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:37.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:37 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:37.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:37 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:02:37.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:37 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:02:37.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:37 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:37.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:37 smithi102 ceph-mon[29849]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:37.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:37 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:37.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:37 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:37.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:37 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:02:37.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:37 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:02:37.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:37 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:02:37.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:37 smithi146 ceph-mon[33158]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:39.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:39 smithi146 ceph-mon[33158]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:39.684 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:02:39 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:02:39.849 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:39 smithi102 ceph-mon[29849]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:39.850 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:39 smithi102 ceph-mon[41371]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:40.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:02:39 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:02:39] "GET /metrics HTTP/1.1" 200 219832 "" "Prometheus/2.43.0" 2024-09-11T08:02:40.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:40 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:02:40.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:40 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:02:40.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:40 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:02:41.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:41 smithi146 ceph-mon[33158]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:41.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:41 smithi102 ceph-mon[41371]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:41.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:41 smithi102 ceph-mon[29849]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:43.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:43 smithi146 ceph-mon[33158]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:43.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:43 smithi102 ceph-mon[41371]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:43.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:43 smithi102 ceph-mon[29849]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:44.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:44 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:02:44.684 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:02:44 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:02:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:02:44.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:44 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:02:44.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:44 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:02:45.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:45 smithi146 ceph-mon[33158]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:45.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:45 smithi102 ceph-mon[29849]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:45.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:45 smithi102 ceph-mon[41371]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:47.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:47 smithi146 ceph-mon[33158]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:47.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:47 smithi102 ceph-mon[29849]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:47.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:47 smithi102 ceph-mon[41371]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:49.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:49 smithi146 ceph-mon[33158]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:49.684 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:02:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:02:49.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:49 smithi102 ceph-mon[41371]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:49.849 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:49 smithi102 ceph-mon[29849]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:50.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:02:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:02:49] "GET /metrics HTTP/1.1" 200 219825 "" "Prometheus/2.43.0" 2024-09-11T08:02:50.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:50 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:02:50.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:50 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:02:50.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:50 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:02:51.436 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:51 smithi102 ceph-mon[29849]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:51.709 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:51 smithi102 ceph-mon[41371]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:51.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:51 smithi146 ceph-mon[33158]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:53.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:53 smithi102 ceph-mon[41371]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:53.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:53 smithi102 ceph-mon[29849]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:53.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:53 smithi146 ceph-mon[33158]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:54.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:02:54 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:02:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:02:55.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:55 smithi102 ceph-mon[41371]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:55.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:55 smithi102 ceph-mon[29849]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:55.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:55 smithi146 ceph-mon[33158]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:57.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:57 smithi102 ceph-mon[41371]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:57.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:57 smithi102 ceph-mon[29849]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:57.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:57 smithi146 ceph-mon[33158]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:02:59.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:59 smithi146 ceph-mon[33158]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:59.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:02:59 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:02:59.684 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:02:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:02:59.850 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:59 smithi102 ceph-mon[41371]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:59.850 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:02:59 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:02:59.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:59 smithi102 ceph-mon[29849]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:02:59.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:02:59 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:03:00.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:02:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:02:59] "GET /metrics HTTP/1.1" 200 219789 "" "Prometheus/2.43.0" 2024-09-11T08:03:00.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:00 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:03:00.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:00 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:03:00.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:00 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:03:01.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:01 smithi102 ceph-mon[41371]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:03:01.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:01 smithi102 ceph-mon[29849]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:03:01.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:01 smithi146 ceph-mon[33158]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:03:03.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:03 smithi102 ceph-mon[29849]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:03:03.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:03 smithi102 ceph-mon[41371]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:03:03.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:03 smithi146 ceph-mon[33158]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:03:04.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:03:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:03:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:03:05.847 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:05 smithi146 ceph-mon[33158]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:03:05.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:05 smithi102 ceph-mon[29849]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:03:05.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:05 smithi102 ceph-mon[41371]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:03:05.914 INFO:tasks.workunit.client.0.smithi102.stderr:Updating files: 99% (12631/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-11T08:03:07.076 INFO:tasks.workunit.client.0.smithi102.stderr:Note: switching to 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'. 2024-09-11T08:03:07.076 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-09-11T08:03:07.076 INFO:tasks.workunit.client.0.smithi102.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-11T08:03:07.076 INFO:tasks.workunit.client.0.smithi102.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-11T08:03:07.076 INFO:tasks.workunit.client.0.smithi102.stderr:state without impacting any branches by switching back to a branch. 2024-09-11T08:03:07.076 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-09-11T08:03:07.077 INFO:tasks.workunit.client.0.smithi102.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-11T08:03:07.077 INFO:tasks.workunit.client.0.smithi102.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-11T08:03:07.077 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-09-11T08:03:07.077 INFO:tasks.workunit.client.0.smithi102.stderr: git switch -c 2024-09-11T08:03:07.077 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-09-11T08:03:07.077 INFO:tasks.workunit.client.0.smithi102.stderr:Or undo this operation with: 2024-09-11T08:03:07.077 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-09-11T08:03:07.077 INFO:tasks.workunit.client.0.smithi102.stderr: git switch - 2024-09-11T08:03:07.077 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-09-11T08:03:07.077 INFO:tasks.workunit.client.0.smithi102.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-11T08:03:07.077 INFO:tasks.workunit.client.0.smithi102.stderr: 2024-09-11T08:03:07.077 INFO:tasks.workunit.client.0.smithi102.stderr:HEAD is now at d0e6828a201 Merge branch 'wip-63919-quincy' of https://github.com/guits/ceph into wip-guits-quincy-2024-09-10-2008-quincy 2024-09-11T08:03:07.082 DEBUG:teuthology.orchestra.run.smithi102:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-11T08:03:07.112 INFO:tasks.workunit.client.0.smithi102.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-11T08:03:07.114 INFO:tasks.workunit.client.0.smithi102.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T08:03:07.114 INFO:tasks.workunit.client.0.smithi102.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-11T08:03:07.829 INFO:tasks.workunit.client.0.smithi102.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-11T08:03:07.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:07 smithi102 ceph-mon[41371]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:03:07.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:07 smithi102 ceph-mon[29849]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:03:07.899 INFO:tasks.workunit.client.0.smithi102.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-11T08:03:07.936 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:07 smithi146 ceph-mon[33158]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:03:07.939 INFO:tasks.workunit.client.0.smithi102.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T08:03:07.940 INFO:tasks.workunit.client.0.smithi102.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T08:03:07.941 INFO:tasks.workunit.client.0.smithi102.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-11T08:03:07.979 INFO:tasks.workunit.client.0.smithi102.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T08:03:07.982 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-11T08:03:07.983 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-11T08:03:08.039 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-09-11T08:03:08.040 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-09-11T08:03:08.040 DEBUG:teuthology.orchestra.run.smithi102:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d0e6828a2016d48cf25ad84064e50742bb1c39b9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-09-11T08:03:08.105 INFO:tasks.workunit.client.0.smithi102.stderr:+ parallel=1 2024-09-11T08:03:08.105 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' '' = --serial ']' 2024-09-11T08:03:08.105 INFO:tasks.workunit.client.0.smithi102.stderr:+ color= 2024-09-11T08:03:08.105 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -t 1 ']' 2024-09-11T08:03:08.105 INFO:tasks.workunit.client.0.smithi102.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-09-11T08:03:08.105 INFO:tasks.workunit.client.0.smithi102.stderr:+ declare -A pids 2024-09-11T08:03:08.105 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.105 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.105 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_aio 2024-09-11T08:03:08.105 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_aio' 2024-09-11T08:03:08.106 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_aio 2024-09-11T08:03:08.106 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.106 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_aio 2024-09-11T08:03:08.107 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80271 2024-09-11T08:03:08.107 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_aio on pid 80271' 2024-09-11T08:03:08.107 INFO:tasks.workunit.client.0.smithi102.stdout:test api_aio on pid 80271 2024-09-11T08:03:08.110 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80271 2024-09-11T08:03:08.110 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.110 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.110 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-09-11T08:03:08.110 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_aio_pp 2024-09-11T08:03:08.110 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_aio_pp' 2024-09-11T08:03:08.110 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_aio_pp 2024-09-11T08:03:08.110 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.110 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.111 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.111 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.111 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.111 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.111 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.111 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.111 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.111 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.111 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.111 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.111 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.111 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.111 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.112 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.112 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.112 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.112 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.112 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.112 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.112 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.112 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.112 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.112 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.112 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.112 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_aio_pp 2024-09-11T08:03:08.112 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80278 2024-09-11T08:03:08.113 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_aio_pp on pid 80278' 2024-09-11T08:03:08.113 INFO:tasks.workunit.client.0.smithi102.stdout:test api_aio_pp on pid 80278 2024-09-11T08:03:08.116 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80278 2024-09-11T08:03:08.116 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.116 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.117 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-09-11T08:03:08.117 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_io 2024-09-11T08:03:08.117 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_io' 2024-09-11T08:03:08.117 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_io 2024-09-11T08:03:08.117 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.117 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.117 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.117 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.117 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.117 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.117 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.117 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.118 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.118 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.118 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.118 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.118 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.118 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.118 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.118 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.118 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.118 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.118 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.118 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.118 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.118 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.119 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.119 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.119 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.119 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.119 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.119 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.119 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.119 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.119 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.119 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.119 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.119 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.119 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.120 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.120 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.120 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.120 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.120 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.120 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.120 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.120 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.120 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.120 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.120 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.120 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_io 2024-09-11T08:03:08.120 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80287 2024-09-11T08:03:08.121 INFO:tasks.workunit.client.0.smithi102.stdout:test api_io on pid 80287 2024-09-11T08:03:08.125 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_io on pid 80287' 2024-09-11T08:03:08.125 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80287 2024-09-11T08:03:08.125 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.125 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-09-11T08:03:08.125 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.125 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_io_pp 2024-09-11T08:03:08.125 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_io_pp' 2024-09-11T08:03:08.125 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_io_pp 2024-09-11T08:03:08.126 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.126 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.126 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.126 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.126 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.126 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.126 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.126 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.126 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.126 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.126 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.126 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.126 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.127 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.127 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.127 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.127 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.127 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.127 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.127 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.127 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.127 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.127 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.127 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.127 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.127 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.128 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.128 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.128 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.128 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.128 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.128 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.128 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.128 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.128 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.128 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.128 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.128 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.128 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.129 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.129 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.129 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.129 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.129 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.129 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.129 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.129 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.129 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.129 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.129 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.129 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.129 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.130 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.130 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.130 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.130 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_io_pp 2024-09-11T08:03:08.130 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80298 2024-09-11T08:03:08.130 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_io_pp on pid 80298' 2024-09-11T08:03:08.130 INFO:tasks.workunit.client.0.smithi102.stdout:test api_io_pp on pid 80298 2024-09-11T08:03:08.136 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80298 2024-09-11T08:03:08.136 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.136 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.136 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-09-11T08:03:08.136 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_asio 2024-09-11T08:03:08.136 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_asio' 2024-09-11T08:03:08.137 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_asio 2024-09-11T08:03:08.137 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.137 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.137 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.137 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.137 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.137 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.137 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.137 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.137 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.137 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.137 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.138 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.138 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.138 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.138 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.138 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.138 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.138 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.139 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.139 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.139 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.139 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.140 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.140 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.140 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.140 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.140 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.140 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.140 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.140 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.140 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.140 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.140 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.140 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.140 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.141 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.141 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.141 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.141 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.141 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.141 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.141 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.141 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.141 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.141 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.141 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.141 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.141 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.142 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.142 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.142 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.142 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.142 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.142 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.142 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.142 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.142 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.142 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.142 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.142 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.142 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.143 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.143 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.143 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.143 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.143 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.143 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.143 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.143 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.143 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.143 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.143 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.143 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.143 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.144 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.144 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_asio 2024-09-11T08:03:08.144 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.144 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.144 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.144 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80313 2024-09-11T08:03:08.144 INFO:tasks.workunit.client.0.smithi102.stdout:test api_asio on pid 80313 2024-09-11T08:03:08.151 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_asio on pid 80313' 2024-09-11T08:03:08.151 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80313 2024-09-11T08:03:08.151 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-09-11T08:03:08.151 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.151 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.152 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.152 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_list 2024-09-11T08:03:08.152 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_list' 2024-09-11T08:03:08.152 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_list 2024-09-11T08:03:08.152 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.152 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.152 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.152 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.152 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.152 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.152 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.152 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.152 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.152 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.153 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.153 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.153 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.153 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.153 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.153 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.153 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.153 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.153 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.153 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.153 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.153 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.154 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.154 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.154 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.154 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.154 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.154 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.154 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.154 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.154 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.154 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.154 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.154 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.154 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.154 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.155 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.155 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.155 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.155 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.155 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.155 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.155 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.155 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.155 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.155 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.155 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.155 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.155 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.155 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.156 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.156 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.156 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.156 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.156 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.156 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.156 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.156 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.156 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.156 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.156 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.156 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.156 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.157 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.157 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.157 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.157 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.157 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.157 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.157 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.157 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.157 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.157 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.157 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.157 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.158 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.158 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.158 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80271/exe 2024-09-11T08:03:08.158 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.158 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.158 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.158 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.158 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.158 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.158 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.158 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.158 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.158 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.158 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.159 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.159 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.159 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.159 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.159 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.159 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.159 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.159 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.159 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.159 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.159 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.159 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.159 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_list 2024-09-11T08:03:08.160 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.160 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80331 2024-09-11T08:03:08.160 INFO:tasks.workunit.client.0.smithi102.stdout:test api_list on pid 80331 2024-09-11T08:03:08.162 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_list on pid 80331' 2024-09-11T08:03:08.162 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80331 2024-09-11T08:03:08.162 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.162 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.162 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-09-11T08:03:08.162 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_lock 2024-09-11T08:03:08.163 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_lock' 2024-09-11T08:03:08.163 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.163 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.163 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.163 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.163 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.163 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.163 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.163 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.163 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.163 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.163 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.163 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.163 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.164 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.164 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.164 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.164 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.164 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.164 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.164 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.164 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.164 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.164 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.164 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.164 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.164 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.164 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.165 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.165 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.165 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.165 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.165 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.165 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.165 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.165 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.165 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.165 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.165 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.165 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.166 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.166 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.166 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.166 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.166 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_aio 2024-09-11T08:03:08.166 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.166 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.166 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.166 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.166 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.166 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.166 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.166 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.166 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.167 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.167 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.167 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.167 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.167 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.167 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.167 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.167 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.167 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.167 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.167 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.167 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.167 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.168 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.168 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.168 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_lock 2024-09-11T08:03:08.168 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.168 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.168 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.168 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.168 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.168 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_aio.log 2024-09-11T08:03:08.168 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.168 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.168 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.168 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.168 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.169 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.169 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.169 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.169 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.169 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.169 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.169 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.169 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.169 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80278/exe 2024-09-11T08:03:08.169 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.169 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.169 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.169 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.169 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.170 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.170 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.170 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.170 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.170 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.170 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.170 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.170 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.170 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.170 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.170 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.170 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.170 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.170 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.171 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_aio: /' 2024-09-11T08:03:08.171 INFO:tasks.workunit.client.0.smithi102.stdout:test api_lock on pid 80351 2024-09-11T08:03:08.176 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.176 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.176 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.176 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.176 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.176 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.177 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_lock 2024-09-11T08:03:08.177 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.177 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80351 2024-09-11T08:03:08.177 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_lock on pid 80351' 2024-09-11T08:03:08.177 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80351 2024-09-11T08:03:08.177 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.177 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.177 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-09-11T08:03:08.177 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.177 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.177 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_lock_pp 2024-09-11T08:03:08.178 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.178 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.178 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.178 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.178 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.178 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_lock_pp' 2024-09-11T08:03:08.178 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.178 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.178 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.178 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.178 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.178 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.179 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.179 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.179 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.179 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.179 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.179 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.179 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_lock_pp 2024-09-11T08:03:08.179 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_aio_pp: /' 2024-09-11T08:03:08.179 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.179 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.179 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.179 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.180 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.180 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_aio_pp 2024-09-11T08:03:08.180 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.180 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.180 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.180 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.180 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.180 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.180 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.180 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.180 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.180 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.181 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.181 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.181 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.181 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.181 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.181 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.181 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.181 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.181 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.181 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.181 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.181 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.181 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.182 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.182 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.182 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.182 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.182 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.182 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.182 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.182 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.182 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.182 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.182 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.182 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.182 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.183 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.183 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.183 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.183 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.183 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.183 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.183 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.183 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.183 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.183 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.183 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.183 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.184 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.184 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.184 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.184 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.184 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.184 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.184 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.184 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.184 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.184 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.184 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.184 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.184 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.185 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.185 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80287/exe 2024-09-11T08:03:08.185 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.185 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.185 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-09-11T08:03:08.185 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.185 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.185 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.185 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.185 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.185 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.185 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.185 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.185 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.186 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.186 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.186 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.186 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.186 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.186 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.186 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.186 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.186 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.186 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.186 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.186 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.187 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.187 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.187 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.187 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.187 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.187 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.187 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.187 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.187 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.187 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.187 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.187 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.187 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.188 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.188 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.188 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.188 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.188 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.188 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.188 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.188 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.188 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.188 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.188 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.189 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.189 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_io: /' 2024-09-11T08:03:08.189 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.189 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.189 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.189 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.189 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.189 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.189 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.189 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.189 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.190 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.190 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.190 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.190 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.190 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.190 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.190 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.190 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.190 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_io 2024-09-11T08:03:08.190 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.190 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.190 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.191 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.191 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.191 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.191 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.191 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.191 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.191 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.191 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.191 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.191 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.191 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.191 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.191 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.191 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_lock_pp 2024-09-11T08:03:08.191 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.192 INFO:tasks.workunit.client.0.smithi102.stdout:test api_lock_pp on pid 80378 2024-09-11T08:03:08.195 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80378 2024-09-11T08:03:08.196 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_lock_pp on pid 80378' 2024-09-11T08:03:08.196 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80378 2024-09-11T08:03:08.196 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.196 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.196 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_io.log 2024-09-11T08:03:08.196 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.196 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-09-11T08:03:08.196 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.196 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.196 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.196 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.197 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.197 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.197 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.197 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.197 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.197 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.197 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.197 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.197 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.197 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_misc 2024-09-11T08:03:08.197 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.197 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.197 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.198 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.198 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.198 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.198 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.198 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_misc' 2024-09-11T08:03:08.198 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_misc 2024-09-11T08:03:08.198 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.198 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.198 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.198 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80298/exe 2024-09-11T08:03:08.198 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.198 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.198 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.199 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.199 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.199 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.199 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.199 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.199 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.199 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.199 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.199 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.199 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.199 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.200 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.200 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.200 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.200 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.200 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.200 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.200 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.200 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.200 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.200 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.200 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.200 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.200 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.201 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.201 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.201 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.201 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.201 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.201 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.201 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.201 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.201 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.201 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.201 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.201 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.202 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.202 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.202 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.202 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.202 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.202 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.202 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.202 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.202 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.202 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.202 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.202 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.203 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.203 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.203 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.203 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.203 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.203 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.203 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.203 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.203 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.203 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.203 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.203 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.203 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.203 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.204 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.204 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.204 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.204 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.204 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.204 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.204 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.204 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.204 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.204 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.204 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.204 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.204 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.204 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.205 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.205 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.205 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.205 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.205 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.205 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.205 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.205 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.205 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.205 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.205 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.205 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.205 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.205 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.206 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_io_pp 2024-09-11T08:03:08.206 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80313/exe 2024-09-11T08:03:08.206 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.206 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.206 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.206 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.206 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.206 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.206 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.206 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.206 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.206 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.206 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.207 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.207 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.207 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.207 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.207 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.207 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.207 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.207 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.207 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.207 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.207 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.207 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.207 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.208 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.208 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.208 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.208 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.208 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.208 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.208 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.208 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.208 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.208 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.208 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.208 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.208 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.209 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.209 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.209 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.209 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.209 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.209 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.209 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.209 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.209 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.209 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.209 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.209 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80331/exe 2024-09-11T08:03:08.209 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.210 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.210 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.210 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.210 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.210 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.210 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.210 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.210 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.210 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.210 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.210 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.210 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.210 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.210 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.211 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.211 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.211 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.211 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.211 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.211 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.211 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_asio.log 2024-09-11T08:03:08.211 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_asio: /' 2024-09-11T08:03:08.211 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.211 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.211 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.211 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.211 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.212 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.212 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.212 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.212 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.212 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.212 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.212 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.212 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_asio 2024-09-11T08:03:08.212 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.212 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.212 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.212 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.212 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.213 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.213 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.213 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.213 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.213 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_list: /' 2024-09-11T08:03:08.213 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_io_pp: /' 2024-09-11T08:03:08.213 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_list.log 2024-09-11T08:03:08.213 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_list 2024-09-11T08:03:08.213 INFO:tasks.workunit.client.0.smithi102.stdout:test api_misc on pid 80415 2024-09-11T08:03:08.219 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_misc 2024-09-11T08:03:08.220 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80415 2024-09-11T08:03:08.220 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_misc on pid 80415' 2024-09-11T08:03:08.220 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80415 2024-09-11T08:03:08.220 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.220 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.220 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-09-11T08:03:08.220 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-09-11T08:03:08.220 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.220 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.220 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.220 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.220 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.221 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.221 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.221 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.221 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.221 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.221 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.221 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.221 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.221 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.221 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.221 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.221 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.222 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.222 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.222 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.222 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.222 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.222 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.222 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.222 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.222 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.222 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.222 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.222 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.223 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.223 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.223 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.223 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.223 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.223 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.223 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.223 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.223 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.223 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.223 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.223 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.223 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.223 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.224 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.224 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.224 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.224 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.224 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.224 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.224 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.224 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.224 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.224 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.224 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.224 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.224 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.225 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.225 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.225 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.225 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.225 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.225 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.225 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.225 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.225 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.225 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.225 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.225 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.225 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.226 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.226 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.226 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.226 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.226 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.226 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.226 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.226 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.226 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.226 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.226 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.226 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.227 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.227 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.227 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.227 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.227 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.227 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.227 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.227 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.227 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.227 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80378/exe 2024-09-11T08:03:08.227 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.227 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.227 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.227 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.228 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.228 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.228 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.228 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.228 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.228 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.228 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.228 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.228 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.228 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.228 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.228 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.229 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.229 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.229 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.229 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.229 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_misc_pp 2024-09-11T08:03:08.229 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-09-11T08:03:08.229 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_misc_pp' 2024-09-11T08:03:08.229 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.229 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_lock_pp 2024-09-11T08:03:08.229 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.229 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.229 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.229 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.229 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.230 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.230 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.230 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.230 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.230 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.230 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.230 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.230 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.230 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.230 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.230 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.230 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.230 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_lock_pp: /' 2024-09-11T08:03:08.231 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.231 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.231 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.231 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.231 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.231 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.231 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.231 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.231 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.231 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.231 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.231 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.231 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.232 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.232 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.232 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.232 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.232 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.232 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.232 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.232 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.232 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.232 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.232 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.232 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.232 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.232 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.233 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80351/exe 2024-09-11T08:03:08.233 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.233 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.233 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.233 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.233 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.233 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.233 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.233 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.233 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.233 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.233 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.233 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.233 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.234 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.234 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.234 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.234 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.234 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.234 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_lock.log 2024-09-11T08:03:08.234 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_misc_pp 2024-09-11T08:03:08.234 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_lock 2024-09-11T08:03:08.234 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_lock: /' 2024-09-11T08:03:08.234 INFO:tasks.workunit.client.0.smithi102.stdout:test api_misc_pp on pid 80476 2024-09-11T08:03:08.235 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_misc_pp 2024-09-11T08:03:08.235 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80476 2024-09-11T08:03:08.236 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_misc_pp on pid 80476' 2024-09-11T08:03:08.236 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80476 2024-09-11T08:03:08.236 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.236 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.236 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_tier_pp 2024-09-11T08:03:08.236 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_tier_pp' 2024-09-11T08:03:08.236 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.236 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-09-11T08:03:08.236 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_tier_pp 2024-09-11T08:03:08.236 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.236 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.236 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_tier_pp 2024-09-11T08:03:08.236 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.237 INFO:tasks.workunit.client.0.smithi102.stdout:test api_tier_pp on pid 80497 2024-09-11T08:03:08.238 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80497 2024-09-11T08:03:08.238 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_tier_pp on pid 80497' 2024-09-11T08:03:08.238 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80497 2024-09-11T08:03:08.238 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.238 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.239 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.239 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.239 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.239 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.239 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-09-11T08:03:08.239 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_pool 2024-09-11T08:03:08.239 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_pool' 2024-09-11T08:03:08.239 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.239 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.239 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.239 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.239 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.239 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.239 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.240 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.240 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.240 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.240 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.240 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.240 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.240 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.240 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.240 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.240 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.240 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.240 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.240 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.241 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.241 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.241 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_pool 2024-09-11T08:03:08.241 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.241 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.241 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.241 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.241 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.241 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.241 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.241 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.241 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.241 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.242 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.242 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.242 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.242 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.242 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.242 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.242 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.242 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.242 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.242 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.242 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.242 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.242 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.243 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.243 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.243 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.243 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.243 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.243 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.243 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.243 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.243 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.243 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.243 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.243 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.243 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.243 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.244 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.244 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.244 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.244 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.244 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.244 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.244 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.244 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80415/exe 2024-09-11T08:03:08.244 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.244 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.244 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.245 INFO:tasks.workunit.client.0.smithi102.stdout:test api_pool on pid 80529 2024-09-11T08:03:08.246 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.246 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.246 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.246 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.247 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.247 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.247 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.247 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.247 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_pool 2024-09-11T08:03:08.247 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80529 2024-09-11T08:03:08.247 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_pool on pid 80529' 2024-09-11T08:03:08.247 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80529 2024-09-11T08:03:08.247 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.247 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.247 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_snapshots 2024-09-11T08:03:08.247 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_snapshots' 2024-09-11T08:03:08.247 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.248 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.248 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.248 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.248 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.248 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.248 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.248 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.248 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.248 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.248 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.248 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.248 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.249 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.249 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.249 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.249 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.249 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.249 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.249 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.249 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.249 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_misc 2024-09-11T08:03:08.249 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_misc: /' 2024-09-11T08:03:08.249 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-09-11T08:03:08.249 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.249 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.250 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.250 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.250 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.250 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.250 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.250 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.250 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.250 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.250 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_snapshots 2024-09-11T08:03:08.250 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.250 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.250 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.250 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.251 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.251 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.251 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.251 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.251 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.251 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.251 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.251 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.251 INFO:tasks.workunit.client.0.smithi102.stdout:test api_snapshots on pid 80554 2024-09-11T08:03:08.253 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_snapshots 2024-09-11T08:03:08.253 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80554 2024-09-11T08:03:08.253 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_snapshots on pid 80554' 2024-09-11T08:03:08.253 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80554 2024-09-11T08:03:08.253 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.254 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.254 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-09-11T08:03:08.254 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_misc.log 2024-09-11T08:03:08.254 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_snapshots_pp 2024-09-11T08:03:08.254 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_snapshots_pp' 2024-09-11T08:03:08.254 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.254 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.254 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.254 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.254 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.254 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.254 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.254 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.255 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.255 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.255 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_snapshots_pp 2024-09-11T08:03:08.255 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.255 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.255 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.255 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.255 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.255 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.255 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.255 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.255 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.255 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.255 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.255 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.256 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.256 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.256 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.256 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.256 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.256 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.256 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.256 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.256 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.256 INFO:tasks.workunit.client.0.smithi102.stdout:test api_snapshots_pp on pid 80566 2024-09-11T08:03:08.259 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_snapshots_pp 2024-09-11T08:03:08.259 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.259 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80566 2024-09-11T08:03:08.259 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.260 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_snapshots_pp on pid 80566' 2024-09-11T08:03:08.260 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80566 2024-09-11T08:03:08.260 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.260 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.260 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.260 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.260 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.260 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.260 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.260 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.260 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.261 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.261 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.261 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.261 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.261 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.261 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.261 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.261 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.261 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.261 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-09-11T08:03:08.261 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.261 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.261 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.262 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.262 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.262 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.262 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.262 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.262 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.262 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.262 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.262 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.262 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.262 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.262 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.262 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.263 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.263 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.263 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.263 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.263 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.263 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.263 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.263 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.263 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.263 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.263 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_stat 2024-09-11T08:03:08.263 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.263 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.264 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.264 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.264 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.264 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.264 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.264 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.264 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.264 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_stat' 2024-09-11T08:03:08.264 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.264 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.264 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.264 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.264 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.265 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.265 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.265 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.265 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.265 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.265 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.265 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.265 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.265 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.265 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.265 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.265 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.265 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.266 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.266 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.266 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.266 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.266 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.266 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.266 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.266 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.266 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.266 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.266 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.266 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.266 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.266 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.267 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.267 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.267 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.267 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.267 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.267 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.267 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.267 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_stat 2024-09-11T08:03:08.267 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.267 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.267 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.267 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.267 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.267 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.268 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.268 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.268 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.268 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.268 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.268 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.268 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.268 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.268 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.268 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.268 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.268 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.268 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.269 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.269 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.269 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.269 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.269 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.269 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.269 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.269 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.269 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_stat 2024-09-11T08:03:08.269 INFO:tasks.workunit.client.0.smithi102.stdout:test api_stat on pid 80599 2024-09-11T08:03:08.272 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80599 2024-09-11T08:03:08.273 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_stat on pid 80599' 2024-09-11T08:03:08.273 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80599 2024-09-11T08:03:08.273 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.273 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.273 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.273 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.273 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.273 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.273 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.273 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.273 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.274 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.274 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.274 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_stat_pp 2024-09-11T08:03:08.274 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_stat_pp' 2024-09-11T08:03:08.274 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.274 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_stat_pp 2024-09-11T08:03:08.274 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.274 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.274 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.274 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-09-11T08:03:08.274 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.274 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.274 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.275 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.275 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.275 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.275 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.275 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.275 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.275 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.275 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.275 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.275 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.275 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.275 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.275 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.276 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.276 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.276 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.276 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.276 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.276 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.276 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.276 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.276 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.276 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.276 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.276 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.276 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.276 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.277 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.277 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.277 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.277 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.277 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.277 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.277 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.277 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.277 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.277 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.277 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.277 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.277 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.277 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.278 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.278 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.278 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.278 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.278 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.278 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.278 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.278 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.278 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.278 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.278 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.278 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.278 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.279 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.279 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.279 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.279 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.279 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.279 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.279 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.279 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.279 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.279 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.279 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.279 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_stat_pp 2024-09-11T08:03:08.279 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.279 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80633 2024-09-11T08:03:08.280 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.280 INFO:tasks.workunit.client.0.smithi102.stdout:test api_stat_pp on pid 80633 2024-09-11T08:03:08.286 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_stat_pp on pid 80633' 2024-09-11T08:03:08.286 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80633 2024-09-11T08:03:08.286 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.286 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.286 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.286 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.286 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.286 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_watch_notify 2024-09-11T08:03:08.286 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.287 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.287 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.287 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_watch_notify' 2024-09-11T08:03:08.287 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.287 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.287 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.287 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.287 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.287 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.287 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.287 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-09-11T08:03:08.287 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.287 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.288 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.288 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.288 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.288 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.288 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.288 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.288 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.288 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.288 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.288 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.288 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.288 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.288 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.289 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.289 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.289 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.289 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.289 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.289 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.289 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.289 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.289 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.289 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.289 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.289 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.290 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.290 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.290 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.290 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.290 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.290 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.290 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.290 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.290 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.290 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.290 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.290 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.290 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.291 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.291 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.291 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.291 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.291 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.291 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.291 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.291 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.291 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.291 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.291 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.291 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.291 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.292 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.292 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.292 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.292 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.292 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.292 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.292 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.292 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.292 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80497/exe 2024-09-11T08:03:08.292 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.292 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.292 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.292 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.293 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.293 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.293 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.293 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.293 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.293 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.293 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.293 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.293 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.293 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.293 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.293 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.294 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.294 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.294 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.294 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.294 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.294 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.294 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.294 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.294 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.294 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.294 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.294 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.295 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.295 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.295 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.295 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.295 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.295 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.295 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.295 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.295 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.295 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.295 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.295 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.295 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.296 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.296 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.296 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.296 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.296 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.296 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.296 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.296 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.296 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.296 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.296 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.296 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.296 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.297 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.297 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.297 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_watch_notify 2024-09-11T08:03:08.297 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.297 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.297 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.297 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.297 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.297 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.297 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.297 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.297 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.297 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.298 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.298 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.298 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.298 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.298 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.298 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.298 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.298 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.298 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.298 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.298 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.298 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.298 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.299 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.299 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.299 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.299 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.299 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.299 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.299 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.299 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.299 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.299 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.299 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.299 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.299 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.299 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.300 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.300 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.300 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.300 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.300 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.300 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.300 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.300 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.300 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.300 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.300 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.300 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.300 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.300 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.300 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.301 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.301 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.301 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.301 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.301 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.301 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.301 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.301 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.301 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.301 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.301 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_tier_pp 2024-09-11T08:03:08.301 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-09-11T08:03:08.301 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_tier_pp: /' 2024-09-11T08:03:08.302 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.302 INFO:tasks.workunit.client.0.smithi102.stdout:test api_watch_notify on pid 80672 2024-09-11T08:03:08.307 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.307 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.307 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.307 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.307 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.307 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.307 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.307 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.307 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.308 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.308 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_watch_notify 2024-09-11T08:03:08.308 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.308 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.308 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.308 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.308 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.308 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.308 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.308 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.308 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.308 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.308 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.309 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.309 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.309 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.309 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.309 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.309 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.309 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80672 2024-09-11T08:03:08.309 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.309 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_watch_notify on pid 80672' 2024-09-11T08:03:08.309 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80672 2024-09-11T08:03:08.309 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.309 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.309 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.310 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.310 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.310 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.310 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.310 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.310 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.310 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.310 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.310 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.310 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.310 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.310 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.310 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80476/exe 2024-09-11T08:03:08.310 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-09-11T08:03:08.311 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.311 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.311 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.311 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.311 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.311 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.311 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.311 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.311 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.311 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.311 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.311 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.311 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.311 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.312 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.312 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.312 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.312 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.312 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.312 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_watch_notify_pp 2024-09-11T08:03:08.312 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.312 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.312 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.312 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.312 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.312 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.312 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.313 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.313 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.313 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.313 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.313 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_watch_notify_pp' 2024-09-11T08:03:08.313 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.313 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.313 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.313 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.313 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.313 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.313 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.314 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.314 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.314 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.314 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.314 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.314 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80566/exe 2024-09-11T08:03:08.314 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_watch_notify_pp 2024-09-11T08:03:08.314 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.314 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.314 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80554/exe 2024-09-11T08:03:08.314 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.314 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.314 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.315 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.315 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.315 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.315 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.315 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.315 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.315 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.315 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80529/exe 2024-09-11T08:03:08.315 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.315 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.315 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.315 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.315 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.316 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.316 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.316 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.316 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.316 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.316 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.316 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.316 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.316 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.316 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.316 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.316 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.316 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.317 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.317 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.317 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.317 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.317 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.317 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.317 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.317 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.317 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.317 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.317 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.318 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.318 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.318 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.318 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.318 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.318 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.318 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.318 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.318 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.318 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.318 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.318 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.318 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.319 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.319 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.319 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.319 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.319 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-09-11T08:03:08.319 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_pool 2024-09-11T08:03:08.319 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_misc_pp: /' 2024-09-11T08:03:08.319 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.319 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.319 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.319 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.319 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.319 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.320 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.320 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.320 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.320 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.320 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-09-11T08:03:08.320 INFO:tasks.workunit.client.0.smithi102.stdout:test api_watch_notify_pp on pid 80707 2024-09-11T08:03:08.327 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_watch_notify_pp 2024-09-11T08:03:08.328 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.328 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.328 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.328 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.328 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.328 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.328 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-09-11T08:03:08.328 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.328 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80707 2024-09-11T08:03:08.328 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_watch_notify_pp on pid 80707' 2024-09-11T08:03:08.328 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80707 2024-09-11T08:03:08.329 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.329 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.329 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.329 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.329 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.329 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_pool: /' 2024-09-11T08:03:08.329 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_misc_pp 2024-09-11T08:03:08.329 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-09-11T08:03:08.329 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_snapshots: /' 2024-09-11T08:03:08.329 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_cmd 2024-09-11T08:03:08.329 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.329 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.329 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_cmd' 2024-09-11T08:03:08.330 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.330 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.330 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.330 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.330 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.330 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.330 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.330 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.330 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.330 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.330 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.330 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.330 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.330 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.331 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.331 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.331 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.331 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.331 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.331 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.331 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.331 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.331 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.331 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.331 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-09-11T08:03:08.331 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_pool.log 2024-09-11T08:03:08.331 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.332 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.332 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_snapshots_pp 2024-09-11T08:03:08.332 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.332 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.332 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.332 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.332 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.332 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.332 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.332 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.332 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.332 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.333 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.333 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.333 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.333 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.333 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.333 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.333 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.333 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.333 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.333 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.333 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.333 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.333 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.334 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.334 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.334 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.334 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.334 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.334 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.334 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.334 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.334 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.334 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.334 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.334 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.335 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.335 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.335 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.335 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.335 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.335 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.335 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_snapshots 2024-09-11T08:03:08.335 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.335 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.335 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.335 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_cmd 2024-09-11T08:03:08.335 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.335 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.336 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.336 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.336 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.336 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.336 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.336 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.336 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.336 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.336 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.336 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.336 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.336 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.337 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.337 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.337 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.337 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.337 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.337 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.337 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.337 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.337 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.337 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.337 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.337 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.338 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.338 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.338 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.338 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.338 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.338 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.338 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.338 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.338 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.338 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.338 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.338 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.338 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.338 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.339 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.339 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.339 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.339 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.339 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.339 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.339 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.339 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.339 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.339 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.339 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.340 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.340 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.340 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.340 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.340 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.340 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.340 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.340 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.340 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.340 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.340 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.340 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.340 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.341 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.341 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.341 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.341 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.344 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.344 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.344 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.344 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.344 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.344 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.344 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.344 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.344 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.345 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.345 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.345 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.345 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.345 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.345 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.345 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.345 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.345 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.345 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.345 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.345 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.345 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.345 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.346 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.346 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.346 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.346 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.346 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.346 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.346 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.346 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.346 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.346 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.346 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.346 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80633/exe 2024-09-11T08:03:08.347 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.347 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.347 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.347 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.347 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.347 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.347 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.347 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.347 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.347 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.347 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.347 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.348 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.348 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.348 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.348 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.348 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.348 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.348 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.348 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.348 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.348 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.348 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.348 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.348 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.349 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.349 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.349 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.349 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.349 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.349 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.349 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.349 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.349 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.349 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_stat_pp: /' 2024-09-11T08:03:08.350 INFO:tasks.workunit.client.0.smithi102.stdout:test api_cmd on pid 80760 2024-09-11T08:03:08.353 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_cmd 2024-09-11T08:03:08.354 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.354 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.354 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.354 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.354 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.354 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.354 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.354 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80760 2024-09-11T08:03:08.354 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_cmd on pid 80760' 2024-09-11T08:03:08.354 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.354 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80760 2024-09-11T08:03:08.354 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.355 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.355 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.355 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.355 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.355 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.355 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.355 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.355 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.355 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.355 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.355 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.355 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.355 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.356 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.356 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.356 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.356 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.356 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.356 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.356 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.356 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80672/exe 2024-09-11T08:03:08.356 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.356 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.356 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.356 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.356 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.357 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.357 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.357 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.357 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.357 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.357 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.357 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-09-11T08:03:08.357 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_cmd_pp 2024-09-11T08:03:08.357 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_cmd_pp' 2024-09-11T08:03:08.357 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-09-11T08:03:08.357 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.357 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_stat_pp 2024-09-11T08:03:08.357 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.357 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.358 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.358 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.358 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-09-11T08:03:08.358 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.358 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.358 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.358 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.358 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.358 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.358 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.358 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.358 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.358 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.358 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.359 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.359 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.359 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.359 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.359 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.359 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.359 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.359 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.359 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.359 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.359 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.359 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.359 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80707/exe 2024-09-11T08:03:08.359 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.360 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.360 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.360 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.360 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.360 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.360 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.360 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.360 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.360 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_watch_notify 2024-09-11T08:03:08.360 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.360 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.360 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.360 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.361 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.361 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.361 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.361 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.361 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.361 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.361 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.361 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.361 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.361 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.361 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.361 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.361 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.362 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.362 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.362 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.362 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.362 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.362 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.362 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.362 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.362 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.362 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_watch_notify: /' 2024-09-11T08:03:08.362 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_watch_notify_pp 2024-09-11T08:03:08.362 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-09-11T08:03:08.362 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-09-11T08:03:08.362 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.362 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.363 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.363 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.363 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.363 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.363 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.363 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.363 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.363 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.363 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.363 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.363 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.363 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.363 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.363 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.364 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.364 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.364 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.364 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.364 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.364 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.364 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.364 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.364 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.364 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.364 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.364 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.364 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.364 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.365 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.365 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_cmd_pp 2024-09-11T08:03:08.365 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.365 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.365 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.365 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.365 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.365 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.365 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.365 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.365 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.365 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.365 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.365 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.366 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_cmd_pp 2024-09-11T08:03:08.366 INFO:tasks.workunit.client.0.smithi102.stdout:test api_cmd_pp on pid 80821 2024-09-11T08:03:08.368 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80821 2024-09-11T08:03:08.368 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_cmd_pp on pid 80821' 2024-09-11T08:03:08.369 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80821 2024-09-11T08:03:08.369 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.369 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.369 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-09-11T08:03:08.369 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.369 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.369 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.369 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.369 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.369 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.369 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.369 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.369 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_service 2024-09-11T08:03:08.369 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.370 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.370 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.370 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.370 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.370 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.370 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.370 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_service' 2024-09-11T08:03:08.370 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.370 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.370 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.370 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.370 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.370 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.371 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.371 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.371 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.371 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.371 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.371 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.371 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.371 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.371 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.371 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.371 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.371 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.371 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.371 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.372 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_service 2024-09-11T08:03:08.372 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.372 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.372 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.372 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.372 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.372 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.372 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.372 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.372 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.372 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.372 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.372 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.373 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.373 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.373 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.373 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.373 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.373 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.373 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.373 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.373 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80599/exe 2024-09-11T08:03:08.373 INFO:tasks.workunit.client.0.smithi102.stdout:test api_service on pid 80853 2024-09-11T08:03:08.377 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_service 2024-09-11T08:03:08.377 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80853 2024-09-11T08:03:08.377 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_service on pid 80853' 2024-09-11T08:03:08.377 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80853 2024-09-11T08:03:08.377 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.377 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.377 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_service_pp 2024-09-11T08:03:08.377 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_service_pp' 2024-09-11T08:03:08.377 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-09-11T08:03:08.377 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.377 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.378 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.378 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.378 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.378 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.378 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.378 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_service_pp 2024-09-11T08:03:08.378 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.378 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.378 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.378 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.378 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.378 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.378 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.378 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.379 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.379 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.379 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.379 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.379 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.379 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.379 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.379 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.379 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.379 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.379 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.379 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.379 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.380 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.380 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.380 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.380 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.380 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.380 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.380 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.380 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.380 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_stat: /' 2024-09-11T08:03:08.380 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_stat 2024-09-11T08:03:08.380 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.380 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.380 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.380 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.381 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.381 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.381 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.381 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.381 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.381 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.381 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.381 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.381 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.381 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.381 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.381 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.382 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.382 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.382 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.382 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.382 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_stat.log 2024-09-11T08:03:08.382 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.382 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.382 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.382 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.382 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.382 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.382 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.383 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.383 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.383 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.383 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.383 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.383 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.383 INFO:tasks.workunit.client.0.smithi102.stdout:test api_service_pp on pid 80899 2024-09-11T08:03:08.385 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_service_pp 2024-09-11T08:03:08.385 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80899 2024-09-11T08:03:08.385 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_service_pp on pid 80899' 2024-09-11T08:03:08.385 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80899 2024-09-11T08:03:08.385 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.385 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.386 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_c_write_operations 2024-09-11T08:03:08.386 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_c_write_operations' 2024-09-11T08:03:08.386 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.386 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.386 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.386 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-09-11T08:03:08.386 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.386 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.386 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.386 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.386 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.386 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.386 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.386 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.387 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.387 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.387 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.387 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.387 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.387 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.387 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.387 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.387 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.387 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.387 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.387 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.388 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.388 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.388 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.388 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.388 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_c_write_operations 2024-09-11T08:03:08.388 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.388 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.388 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.388 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.388 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.388 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.388 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.389 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.389 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.389 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.389 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.389 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.389 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.389 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.389 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.389 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.389 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.389 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.389 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.389 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.389 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.389 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.390 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.390 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.390 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.390 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.390 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.390 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.390 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.390 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.390 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.390 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.390 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.390 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.390 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.390 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.391 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.391 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.391 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.391 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.391 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.391 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.391 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.391 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.391 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.391 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.391 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.391 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.391 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.391 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.392 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.392 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.392 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.392 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.392 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.392 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.392 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.392 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.392 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.392 INFO:tasks.workunit.client.0.smithi102.stdout:test api_c_write_operations on pid 80937 2024-09-11T08:03:08.396 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_c_write_operations 2024-09-11T08:03:08.396 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80760/exe 2024-09-11T08:03:08.396 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80937 2024-09-11T08:03:08.396 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_c_write_operations on pid 80937' 2024-09-11T08:03:08.396 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80937 2024-09-11T08:03:08.396 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.396 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.396 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-09-11T08:03:08.396 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.396 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.396 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.397 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.397 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.397 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.397 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.397 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.397 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.397 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.397 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.397 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.397 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.397 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.397 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.397 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.397 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.398 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.398 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.398 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.398 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.398 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.398 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.398 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.398 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.398 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.398 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.398 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.398 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.398 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.399 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.399 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.399 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.399 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.399 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.399 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.399 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.399 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.399 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.399 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.399 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_cmd.log 2024-09-11T08:03:08.399 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.399 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.399 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.400 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.400 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.400 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.400 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_c_read_operations 2024-09-11T08:03:08.400 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.400 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.400 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.400 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.400 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.400 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.400 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_c_read_operations' 2024-09-11T08:03:08.400 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_c_read_operations 2024-09-11T08:03:08.400 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.400 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.401 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.401 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_cmd: /' 2024-09-11T08:03:08.401 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.401 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.401 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.401 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.401 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.401 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.401 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_cmd 2024-09-11T08:03:08.401 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.401 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.401 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.401 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.401 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.402 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.402 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.402 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.402 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.402 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.402 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.402 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.402 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.402 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80821/exe 2024-09-11T08:03:08.402 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.402 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.402 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.403 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.403 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.403 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.403 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.403 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.403 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.403 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.403 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.403 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.403 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.403 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.403 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.403 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.403 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.403 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.404 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.404 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.404 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.404 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.404 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.404 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.404 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.404 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.404 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.404 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.404 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.404 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.404 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.405 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.405 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.405 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.405 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.405 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.405 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.405 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.405 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.405 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_cmd_pp 2024-09-11T08:03:08.405 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-09-11T08:03:08.406 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.406 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-09-11T08:03:08.406 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.406 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.406 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.406 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.406 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.406 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.406 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.406 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.406 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.406 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.406 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.407 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.407 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.407 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.407 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.407 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.407 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.407 INFO:tasks.workunit.client.0.smithi102.stdout:test api_c_read_operations on pid 80981 2024-09-11T08:03:08.410 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_c_read_operations 2024-09-11T08:03:08.410 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.410 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.411 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.411 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.411 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.411 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.411 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.411 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80981 2024-09-11T08:03:08.411 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_c_read_operations on pid 80981' 2024-09-11T08:03:08.411 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80981 2024-09-11T08:03:08.411 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.411 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.411 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.412 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.412 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.412 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.412 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.412 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.412 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.412 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.412 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.412 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.412 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.412 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.412 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.412 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.412 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.413 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.413 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.413 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s api_cls_remote_reads 2024-09-11T08:03:08.413 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.413 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.413 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.413 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.413 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.413 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.413 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.413 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.413 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' api_cls_remote_reads' 2024-09-11T08:03:08.413 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.413 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.414 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.414 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.414 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.414 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-09-11T08:03:08.414 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.414 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.414 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo api_cls_remote_reads 2024-09-11T08:03:08.414 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.414 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.414 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.414 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.414 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.414 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.415 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.415 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.415 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.415 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.415 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.415 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.415 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.415 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.415 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.415 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.415 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.415 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.415 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.416 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=api_cls_remote_reads 2024-09-11T08:03:08.416 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.416 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.416 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80992 2024-09-11T08:03:08.416 INFO:tasks.workunit.client.0.smithi102.stdout:test api_cls_remote_reads on pid 80992 2024-09-11T08:03:08.420 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test api_cls_remote_reads on pid 80992' 2024-09-11T08:03:08.420 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=80992 2024-09-11T08:03:08.420 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.420 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.420 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.420 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.420 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.420 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.420 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.420 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.420 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.420 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s list_parallel 2024-09-11T08:03:08.420 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.421 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.421 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.421 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.421 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.421 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.421 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.421 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.421 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.421 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.421 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.421 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.421 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' list_parallel' 2024-09-11T08:03:08.421 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2024-09-11T08:03:08.421 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.421 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.422 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.422 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.422 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.422 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.422 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.422 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.422 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.422 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.422 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.422 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.422 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.422 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.422 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.422 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.423 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo list_parallel 2024-09-11T08:03:08.423 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.423 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.423 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.423 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.423 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.423 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.423 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80899/exe 2024-09-11T08:03:08.423 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.423 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.423 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.423 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.423 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.423 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.424 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.424 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.424 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.424 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.424 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.424 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.424 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.424 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.424 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.424 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.424 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.424 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.424 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.424 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.425 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.425 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.425 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.425 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.425 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.425 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.425 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.425 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.425 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.425 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.425 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.425 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.425 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.425 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.426 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.426 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.426 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.426 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.426 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.426 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.426 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.426 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.426 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.426 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.426 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.426 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.426 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.427 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.427 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.427 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.427 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.427 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.427 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.427 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.427 INFO:tasks.workunit.client.0.smithi102.stdout:test list_parallel on pid 81009 2024-09-11T08:03:08.433 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=list_parallel 2024-09-11T08:03:08.433 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=81009 2024-09-11T08:03:08.433 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test list_parallel on pid 81009' 2024-09-11T08:03:08.433 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=81009 2024-09-11T08:03:08.433 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.434 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.434 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-09-11T08:03:08.434 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_service_pp: /' 2024-09-11T08:03:08.434 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.434 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.434 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.434 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.434 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.434 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_service_pp 2024-09-11T08:03:08.434 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-09-11T08:03:08.434 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.434 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.434 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.435 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.435 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.435 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.435 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.435 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.435 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.435 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.435 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.435 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.435 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.435 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.435 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.435 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.435 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.435 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.436 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.436 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.436 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.436 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.436 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.436 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.436 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.436 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.436 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.436 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.436 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.436 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.436 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.437 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.437 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.437 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.437 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.437 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.437 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.437 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.437 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.437 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.437 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.437 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.437 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.437 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.437 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.438 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.438 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.438 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.438 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.438 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.438 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.438 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.438 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.438 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.438 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.438 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.438 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.438 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.439 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.439 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.439 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.439 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.439 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.439 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.439 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.439 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.439 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.439 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.439 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.439 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.439 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.439 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.440 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s open_pools_parallel 2024-09-11T08:03:08.440 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' open_pools_parallel' 2024-09-11T08:03:08.440 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.440 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo open_pools_parallel 2024-09-11T08:03:08.440 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80937/exe 2024-09-11T08:03:08.440 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.440 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.440 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.440 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.440 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.440 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.440 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.440 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.440 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.440 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.441 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.441 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.441 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.441 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.441 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.441 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.441 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.441 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.441 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.441 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.441 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.441 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.441 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.442 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.442 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.442 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.442 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.442 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.442 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.442 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.442 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.442 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.442 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.442 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.442 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.442 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.442 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.443 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.443 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.443 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.443 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.443 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80853/exe 2024-09-11T08:03:08.443 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.443 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.443 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.443 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.443 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.443 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.443 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.443 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.444 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.444 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.444 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.444 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.444 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.444 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.444 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.444 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.444 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.444 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.444 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.444 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.445 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.445 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.445 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.445 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.445 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.445 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.445 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.445 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.445 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.445 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-09-11T08:03:08.445 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.445 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.446 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.446 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.446 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.446 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.446 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.446 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.446 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.446 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.446 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.446 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.446 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.446 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_c_write_operations 2024-09-11T08:03:08.446 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.446 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.447 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.447 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.447 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.447 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.447 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.447 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.447 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.447 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.447 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-09-11T08:03:08.447 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_service 2024-09-11T08:03:08.447 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.447 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.447 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.447 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.448 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.448 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.448 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.448 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.448 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.448 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_service: /' 2024-09-11T08:03:08.448 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.448 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.448 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.448 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.448 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.448 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.448 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.448 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.449 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.449 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.449 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.449 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.449 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.449 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.449 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.449 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.449 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.449 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.449 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.449 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.449 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.449 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.449 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.450 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.450 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.450 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.450 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_service.log 2024-09-11T08:03:08.450 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.450 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.450 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.450 INFO:tasks.workunit.client.0.smithi102.stdout:test open_pools_parallel on pid 81063 2024-09-11T08:03:08.453 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.453 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.453 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.453 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.453 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.453 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.453 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.453 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.454 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.454 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.454 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.454 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=open_pools_parallel 2024-09-11T08:03:08.454 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.454 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.454 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.454 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=81063 2024-09-11T08:03:08.454 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test open_pools_parallel on pid 81063' 2024-09-11T08:03:08.454 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=81063 2024-09-11T08:03:08.454 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-11T08:03:08.454 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.454 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.455 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.455 INFO:tasks.workunit.client.0.smithi102.stderr:++ printf %25s delete_pools_parallel 2024-09-11T08:03:08.455 INFO:tasks.workunit.client.0.smithi102.stderr:+ r=' delete_pools_parallel' 2024-09-11T08:03:08.455 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-09-11T08:03:08.455 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.455 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.455 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.455 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.455 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.455 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.455 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.455 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.455 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.455 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.456 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.456 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.456 INFO:tasks.workunit.client.0.smithi102.stderr:++ awk '{print $1}' 2024-09-11T08:03:08.456 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.456 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.456 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.456 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.456 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.456 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.456 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.456 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.456 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.456 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.457 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.457 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.457 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.457 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.457 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.457 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.457 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.457 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.457 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.457 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.457 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.457 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.457 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.458 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.458 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.458 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.458 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.458 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.458 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.458 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.458 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.458 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.458 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.458 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.458 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.458 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.458 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.458 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.459 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.459 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.459 INFO:tasks.workunit.client.0.smithi102.stderr:++ echo delete_pools_parallel 2024-09-11T08:03:08.459 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.459 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.459 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.459 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.459 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.459 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.459 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.459 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.459 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.459 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.459 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.459 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.459 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.460 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.460 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.460 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.460 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.460 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.460 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.460 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.460 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.460 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.460 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.460 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.460 INFO:tasks.workunit.client.0.smithi102.stdout:test delete_pools_parallel on pid 81097 2024-09-11T08:03:08.468 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.468 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.468 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.468 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.468 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.468 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.468 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.469 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.469 INFO:tasks.workunit.client.0.smithi102.stderr:+ ff=delete_pools_parallel 2024-09-11T08:03:08.469 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.469 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.469 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.469 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.469 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.469 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.469 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.469 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=81097 2024-09-11T08:03:08.469 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo 'test delete_pools_parallel on pid 81097' 2024-09-11T08:03:08.469 INFO:tasks.workunit.client.0.smithi102.stderr:+ pids[$f]=81097 2024-09-11T08:03:08.469 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.469 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.469 INFO:tasks.workunit.client.0.smithi102.stderr:+ ret=0 2024-09-11T08:03:08.470 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.470 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.470 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.470 INFO:tasks.workunit.client.0.smithi102.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-09-11T08:03:08.470 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 1 -eq 1 ']' 2024-09-11T08:03:08.470 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.470 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:03:08.470 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80497 2024-09-11T08:03:08.470 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80497 2024-09-11T08:03:08.470 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80992/exe 2024-09-11T08:03:08.470 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -f /etc/bashrc ']' 2024-09-11T08:03:08.470 INFO:tasks.workunit.client.0.smithi102.stderr:+ . /etc/bashrc 2024-09-11T08:03:08.470 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.470 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -z '' ']' 2024-09-11T08:03:08.471 INFO:tasks.workunit.client.0.smithi102.stderr:++ BASHRCSOURCED=Y 2024-09-11T08:03:08.471 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.471 INFO:tasks.workunit.client.0.smithi102.stderr:++ shopt -q login_shell 2024-09-11T08:03:08.471 INFO:tasks.workunit.client.0.smithi102.stderr:+++ umask 2024-09-11T08:03:08.471 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' 0022 -eq 0 ']' 2024-09-11T08:03:08.471 INFO:tasks.workunit.client.0.smithi102.stderr:++ SHELL=/bin/bash 2024-09-11T08:03:08.471 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.471 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-11T08:03:08.471 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.471 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-11T08:03:08.471 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.471 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.471 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.471 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-11T08:03:08.472 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.472 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.472 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-11T08:03:08.472 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.472 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.472 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.472 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-11T08:03:08.472 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.472 INFO:tasks.workunit.client.0.smithi102.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-11T08:03:08.472 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.472 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.472 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-11T08:03:08.472 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.472 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-11T08:03:08.473 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.473 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.473 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.473 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.473 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2024-09-11T08:03:08.473 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_cls_remote_reads 2024-09-11T08:03:08.473 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.473 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.473 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.473 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.473 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.473 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.473 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.473 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.474 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.474 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.474 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.474 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.474 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.474 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.474 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.474 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.474 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.474 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.474 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.474 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.474 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/81009/exe 2024-09-11T08:03:08.474 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.475 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.475 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.475 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.475 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.475 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.475 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.475 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.475 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.475 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.475 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.475 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.475 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.475 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'grep=grep --color=auto' 2024-09-11T08:03:08.475 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.476 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-11T08:03:08.476 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-11T08:03:08.476 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.476 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-11T08:03:08.476 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.476 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.476 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorls.sh 2024-09-11T08:03:08.476 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.476 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' '!' -t 0 ']' 2024-09-11T08:03:08.476 INFO:tasks.workunit.client.0.smithi102.stderr:+++ return 2024-09-11T08:03:08.476 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.476 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.476 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-11T08:03:08.476 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.477 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-11T08:03:08.477 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export S_COLORS=auto 2024-09-11T08:03:08.477 INFO:tasks.workunit.client.0.smithi102.stderr:+++ S_COLORS=auto 2024-09-11T08:03:08.477 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.477 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-11T08:03:08.477 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.477 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-11T08:03:08.477 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.477 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_list_parallel.log 2024-09-11T08:03:08.477 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_list_parallel 2024-09-11T08:03:08.477 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ list_parallel: /' 2024-09-11T08:03:08.477 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.477 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.478 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.478 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.478 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.478 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.478 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.478 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.478 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.478 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.478 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.478 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.479 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.479 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.479 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.479 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.479 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.479 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.479 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.479 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.479 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.479 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2024-09-11T08:03:08.479 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.479 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.479 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.480 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.480 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.480 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.480 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.480 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.480 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.480 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-11T08:03:08.480 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-11T08:03:08.480 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-11T08:03:08.480 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.480 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-11T08:03:08.481 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.481 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-11T08:03:08.481 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-11T08:03:08.481 INFO:tasks.workunit.client.0.smithi102.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-11T08:03:08.481 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.481 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.481 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.481 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.481 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.481 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.481 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.482 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.482 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.482 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.482 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.482 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.482 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.482 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.482 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.482 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.482 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.482 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-11T08:03:08.482 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-11T08:03:08.483 INFO:tasks.workunit.client.0.smithi102.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-11T08:03:08.483 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.483 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.483 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-11T08:03:08.483 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.483 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-11T08:03:08.483 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-11T08:03:08.483 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.483 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-11T08:03:08.483 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.483 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/gawk.sh 2024-09-11T08:03:08.483 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.484 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-11T08:03:08.484 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.484 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/lang.sh 2024-09-11T08:03:08.484 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup 2024-09-11T08:03:08.484 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/80981/exe 2024-09-11T08:03:08.484 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.484 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.484 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.484 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.484 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.484 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/locale 2024-09-11T08:03:08.484 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.485 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.485 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-11T08:03:08.485 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.485 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-11T08:03:08.485 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-11T08:03:08.485 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.485 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.485 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.485 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.485 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.485 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.485 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.485 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.486 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.486 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.486 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.486 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.486 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.486 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.486 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.486 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.486 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.486 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.486 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.486 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.486 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.486 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.487 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.487 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.487 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.487 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.487 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.487 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.487 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.487 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.487 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.487 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.487 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_api_c_read_operations 2024-09-11T08:03:08.487 INFO:tasks.workunit.client.0.smithi102.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-11T08:03:08.487 INFO:tasks.workunit.client.0.smithi102.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-11T08:03:08.488 INFO:tasks.workunit.client.0.smithi102.stderr:++++ export LANG=en_US.UTF-8 2024-09-11T08:03:08.488 INFO:tasks.workunit.client.0.smithi102.stderr:++++ LANG=en_US.UTF-8 2024-09-11T08:03:08.488 INFO:tasks.workunit.client.0.smithi102.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-11T08:03:08.488 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-11T08:03:08.488 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.488 INFO:tasks.workunit.client.0.smithi102.stderr:+++ LANG=en_US.UTF-8 2024-09-11T08:03:08.488 INFO:tasks.workunit.client.0.smithi102.stderr:+++ unset LANG_backup config 2024-09-11T08:03:08.488 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n '' ']' 2024-09-11T08:03:08.488 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-11T08:03:08.488 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' dumb = linux ']' 2024-09-11T08:03:08.488 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.488 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-11T08:03:08.488 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.489 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/less.sh 2024-09-11T08:03:08.489 INFO:tasks.workunit.client.0.smithi102.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-11T08:03:08.489 INFO:tasks.workunit.client.0.smithi102.stderr:++ for i in /etc/profile.d/*.sh 2024-09-11T08:03:08.489 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-11T08:03:08.489 INFO:tasks.workunit.client.0.smithi102.stderr:++ '[' '' ']' 2024-09-11T08:03:08.489 INFO:tasks.workunit.client.0.smithi102.stderr:++ . /etc/profile.d/which2.sh 2024-09-11T08:03:08.489 INFO:tasks.workunit.client.0.smithi102.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-11T08:03:08.489 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/81097/exe 2024-09-11T08:03:08.489 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.489 INFO:tasks.workunit.client.0.smithi102.stderr:+++++ readlink /proc/81063/exe 2024-09-11T08:03:08.489 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-09-11T08:03:08.489 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.489 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.490 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.490 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.490 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.490 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.490 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.490 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.490 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.490 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.490 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_delete_pools_parallel 2024-09-11T08:03:08.490 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-09-11T08:03:08.490 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-09-11T08:03:08.490 INFO:tasks.workunit.client.0.smithi102.stderr:++++ basename /usr/bin/bash 2024-09-11T08:03:08.490 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-09-11T08:03:08.491 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_declare='declare -f' 2024-09-11T08:03:08.491 INFO:tasks.workunit.client.0.smithi102.stderr:+++ which_opt=-f 2024-09-11T08:03:08.491 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export which_declare 2024-09-11T08:03:08.491 INFO:tasks.workunit.client.0.smithi102.stderr:+++ export -f which 2024-09-11T08:03:08.491 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset i 2024-09-11T08:03:08.491 INFO:tasks.workunit.client.0.smithi102.stderr:++ unset -f pathmunge 2024-09-11T08:03:08.491 INFO:tasks.workunit.client.0.smithi102.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-11T08:03:08.491 INFO:tasks.workunit.client.0.smithi102.stderr:+ export PATH 2024-09-11T08:03:08.491 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-11T08:03:08.491 INFO:tasks.workunit.client.0.smithi102.stderr:+ unset rc 2024-09-11T08:03:08.491 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph_test_rados_open_pools_parallel 2024-09-11T08:03:08.491 INFO:tasks.workunit.client.0.smithi102.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-09-11T08:03:08.491 INFO:tasks.workunit.client.0.smithi102.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-09-11T08:03:09.684 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:03:09 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:03:10.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:03:09 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:03:09] "GET /metrics HTTP/1.1" 200 219789 "" "Prometheus/2.43.0" 2024-09-11T08:03:12.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:11 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/929970514' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:12.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:11 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2446791414' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-09-11T08:03:12.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:11 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/929970514' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:12.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:11 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2446791414' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-09-11T08:03:12.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:11 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/929970514' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:12.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:11 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2446791414' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-09-11T08:03:14.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:13 smithi146 ceph-mon[33158]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:03:14.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3315537506' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi102-80338-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:14.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2139821986' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi102-80356-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:14.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:13 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:03:14.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:13 smithi146 ceph-mon[33158]: pgmap v64: 196 pgs: 57 creating+peering, 7 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:03:14.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:13 smithi146 ceph-mon[33158]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T08:03:14.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:13 smithi146 ceph-mon[33158]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:14.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:13 smithi146 ceph-mon[33158]: from='client.24664 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi102-80338-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:14.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:13 smithi146 ceph-mon[33158]: from='client.24676 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi102-80356-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:14.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:13 smithi102 ceph-mon[41371]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:03:14.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3315537506' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi102-80338-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:14.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2139821986' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi102-80356-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:14.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:13 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:03:14.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:13 smithi102 ceph-mon[41371]: pgmap v64: 196 pgs: 57 creating+peering, 7 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:03:14.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:13 smithi102 ceph-mon[41371]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T08:03:14.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:13 smithi102 ceph-mon[41371]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:14.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:13 smithi102 ceph-mon[41371]: from='client.24664 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi102-80338-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:14.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:13 smithi102 ceph-mon[41371]: from='client.24676 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi102-80356-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:14.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:13 smithi102 ceph-mon[29849]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:03:14.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3315537506' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi102-80338-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:14.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2139821986' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi102-80356-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:14.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:13 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:03:14.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:13 smithi102 ceph-mon[29849]: pgmap v64: 196 pgs: 57 creating+peering, 7 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:03:14.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:13 smithi102 ceph-mon[29849]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T08:03:14.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:13 smithi102 ceph-mon[29849]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:14.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:13 smithi102 ceph-mon[29849]: from='client.24664 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi102-80338-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:14.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:13 smithi102 ceph-mon[29849]: from='client.24676 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi102-80356-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:14.925 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:03:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:03:15.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3312600444' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/425000704' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi102-80667-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3089930629' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi102-80709-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3304252508' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi102-80372-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3482055739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi102-80395-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2599955042' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi102-80461-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1241821602' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi102-80697-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3595409683' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi102-80885-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/528384104' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi102-80429-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/435504180' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi102-81150-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi102-80411-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: pgmap v66: 612 pgs: 57 creating+peering, 423 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:03:15.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/929970514' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:15.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/950753030' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-11T08:03:15.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T08:03:15.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:15.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.24730 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.24761 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi102-80885-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.24739 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi102-80667-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.24695 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi102-80429-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.24742 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi102-80709-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.24830 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi102-81150-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.24685 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi102-80372-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.24692 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi102-80411-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.24688 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi102-80395-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.24709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi102-80461-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.24733 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi102-80697-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.24664 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi102-80338-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:15.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.24676 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi102-80356-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:15.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T08:03:15.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1522090914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi102-80756-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3219604059' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81109-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2673942108' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi102-80768-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1055399853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3682303591' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80973-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3598441879' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80948-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:15.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.24745 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.24757 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi102-80756-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/929970514' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:15.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.24814 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81109-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.24754 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi102-80768-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.24790 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.24844 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80973-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.24865 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80948-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.189 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:15.189 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: pgmap v68: 900 pgs: 57 creating+peering, 711 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:03:15.189 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:03:15.189 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:14 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:03:15.248 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-09-11T08:03:15.248 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [----------] Global test environment set-up. 2024-09-11T08:03:15.248 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [----------] 12 tests from AsioRados 2024-09-11T08:03:15.248 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-09-11T08:03:15.248 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (1 ms) 2024-09-11T08:03:15.248 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-09-11T08:03:15.248 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (1 ms) 2024-09-11T08:03:15.248 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-09-11T08:03:15.248 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (0 ms) 2024-09-11T08:03:15.248 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-09-11T08:03:15.248 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (7 ms) 2024-09-11T08:03:15.248 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-09-11T08:03:15.248 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (2 ms) 2024-09-11T08:03:15.249 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-09-11T08:03:15.249 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (3 ms) 2024-09-11T08:03:15.249 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-09-11T08:03:15.249 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (1 ms) 2024-09-11T08:03:15.249 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-09-11T08:03:15.249 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (1 ms) 2024-09-11T08:03:15.249 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-09-11T08:03:15.249 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (1 ms) 2024-09-11T08:03:15.249 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-09-11T08:03:15.249 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (2 ms) 2024-09-11T08:03:15.249 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-09-11T08:03:15.249 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (2 ms) 2024-09-11T08:03:15.249 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-09-11T08:03:15.249 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (4 ms) 2024-09-11T08:03:15.249 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [----------] 12 tests from AsioRados (25 ms total) 2024-09-11T08:03:15.249 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: 2024-09-11T08:03:15.250 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [----------] Global test environment tear-down 2024-09-11T08:03:15.250 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (7073 ms total) 2024-09-11T08:03:15.250 INFO:tasks.workunit.client.0.smithi102.stdout: api_asio: [ PASSED ] 12 tests. 2024-09-11T08:03:15.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3312600444' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/425000704' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi102-80667-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3089930629' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi102-80709-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3304252508' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi102-80372-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3482055739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi102-80395-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2599955042' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi102-80461-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1241821602' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi102-80697-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3595409683' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi102-80885-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/528384104' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi102-80429-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/435504180' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi102-81150-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi102-80411-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: pgmap v66: 612 pgs: 57 creating+peering, 423 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:03:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/929970514' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/950753030' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-11T08:03:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T08:03:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.24730 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.24761 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi102-80885-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3312600444' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/425000704' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi102-80667-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3089930629' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi102-80709-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3304252508' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi102-80372-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3482055739' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi102-80395-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2599955042' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi102-80461-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1241821602' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi102-80697-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3595409683' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi102-80885-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/528384104' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi102-80429-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/435504180' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi102-81150-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi102-80411-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: pgmap v66: 612 pgs: 57 creating+peering, 423 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:03:15.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/929970514' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:15.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/950753030' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-11T08:03:15.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T08:03:15.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:15.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.24730 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.24761 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi102-80885-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.24739 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi102-80667-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.24695 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi102-80429-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.24742 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi102-80709-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.24830 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi102-81150-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.24685 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi102-80372-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.24692 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi102-80411-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.24688 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi102-80395-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.24709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi102-80461-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.24733 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi102-80697-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.24664 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi102-80338-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:15.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.24676 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi102-80356-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:15.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T08:03:15.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1522090914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi102-80756-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3219604059' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81109-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.401 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2673942108' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi102-80768-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.401 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1055399853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.401 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3682303591' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80973-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.24739 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi102-80667-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.24695 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi102-80429-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.24742 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi102-80709-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.24830 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi102-81150-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.24685 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi102-80372-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.24692 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi102-80411-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.24688 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi102-80395-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.24709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi102-80461-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.24733 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi102-80697-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.24664 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi102-80338-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:15.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.24676 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi102-80356-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:15.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T08:03:15.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1522090914' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi102-80756-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3219604059' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81109-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2673942108' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi102-80768-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1055399853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3682303591' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80973-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.403 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3598441879' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80948-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.403 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:15.403 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.24745 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.403 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.24757 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi102-80756-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.403 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/929970514' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:15.403 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.24814 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81109-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.403 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.24754 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi102-80768-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.403 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.24790 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.403 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.24844 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80973-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.403 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.24865 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80948-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.403 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:15.404 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: pgmap v68: 900 pgs: 57 creating+peering, 711 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:03:15.404 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:03:15.404 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:03:15.404 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3598441879' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80948-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.404 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:15.404 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.24745 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.404 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.24757 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi102-80756-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.404 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/929970514' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:15.404 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.24814 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81109-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.404 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.24754 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi102-80768-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.404 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.24790 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.404 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.24844 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80973-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.405 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.24865 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80948-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:15.405 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:15.405 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: pgmap v68: 900 pgs: 57 creating+peering, 711 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:03:15.405 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:03:15.405 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:14 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:03:15.531 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-09-11T08:03:15.531 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [----------] Global test environment set-up. 2024-09-11T08:03:15.532 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [----------] 7 tests from LibRadosList 2024-09-11T08:03:15.532 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-09-11T08:03:15.532 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [ OK ] LibRadosList.ListObjects (312 ms) 2024-09-11T08:03:15.532 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-09-11T08:03:15.532 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (29 ms) 2024-09-11T08:03:15.532 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-09-11T08:03:15.532 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: myset foo1,foo2,foo3 2024-09-11T08:03:15.532 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: foo1 2024-09-11T08:03:15.532 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: foo2 2024-09-11T08:03:15.532 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: foo3 2024-09-11T08:03:15.532 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: myset foo1,foo4,foo5 2024-09-11T08:03:15.532 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: foo4 2024-09-11T08:03:15.533 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: foo5 2024-09-11T08:03:15.533 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: foo1 2024-09-11T08:03:15.533 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: myset foo6,foo7 2024-09-11T08:03:15.533 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: foo7 2024-09-11T08:03:15.533 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: foo6 2024-09-11T08:03:15.533 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-09-11T08:03:15.533 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: ns1:foo4 2024-09-11T08:03:15.533 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: ns1:foo5 2024-09-11T08:03:15.533 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: ns2:foo7 2024-09-11T08:03:15.533 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: ns2:foo6 2024-09-11T08:03:15.533 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: ns1:foo1 2024-09-11T08:03:15.533 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: :foo1 2024-09-11T08:03:15.533 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: :foo2 2024-09-11T08:03:15.534 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: :foo3 2024-09-11T08:03:15.534 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (101 ms) 2024-09-11T08:03:15.534 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-09-11T08:03:15.534 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 1 0 2024-09-11T08:03:15.534 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 10 0 2024-09-11T08:03:15.534 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 13 0 2024-09-11T08:03:15.534 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 7 0 2024-09-11T08:03:15.534 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 14 0 2024-09-11T08:03:15.534 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 0 0 2024-09-11T08:03:15.534 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 15 0 2024-09-11T08:03:15.534 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 11 0 2024-09-11T08:03:15.534 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 5 0 2024-09-11T08:03:15.534 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 8 0 2024-09-11T08:03:15.534 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 6 0 2024-09-11T08:03:15.535 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 3 0 2024-09-11T08:03:15.535 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 4 0 2024-09-11T08:03:15.535 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 12 0 2024-09-11T08:03:15.535 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 9 0 2024-09-11T08:03:15.535 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 2 0 2024-09-11T08:03:15.535 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-09-11T08:03:15.535 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (64 ms) 2024-09-11T08:03:15.535 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-09-11T08:03:15.535 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: x cursor=MIN 2024-09-11T08:03:15.535 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > oid=1 cursor=9:02547ec2:::1:head 2024-09-11T08:03:15.535 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > oid=10 cursor=9:52ea6a34:::10:head 2024-09-11T08:03:15.535 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > oid=13 cursor=9:566253c9:::13:head 2024-09-11T08:03:15.535 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > oid=7 cursor=9:5c6b0b28:::7:head 2024-09-11T08:03:15.535 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > oid=14 cursor=9:62a1935d:::14:head 2024-09-11T08:03:15.536 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > oid=0 cursor=9:6cac518f:::0:head 2024-09-11T08:03:15.536 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > oid=15 cursor=9:863748b0:::15:head 2024-09-11T08:03:15.536 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > oid=11 cursor=9:89d3ae78:::11:head 2024-09-11T08:03:15.536 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > oid=5 cursor=9:b29083e3:::5:head 2024-09-11T08:03:15.536 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > oid=8 cursor=9:bd63b0f1:::8:head 2024-09-11T08:03:15.536 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > oid=6 cursor=9:c4fdafeb:::6:head 2024-09-11T08:03:15.536 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > oid=3 cursor=9:cfc208b3:::3:head 2024-09-11T08:03:15.536 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > oid=4 cursor=9:d83876eb:::4:head 2024-09-11T08:03:15.536 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > oid=12 cursor=9:de5d7c5f:::12:head 2024-09-11T08:03:15.536 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > oid=9 cursor=9:e960b815:::9:head 2024-09-11T08:03:15.536 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > oid=2 cursor=9:f905c69b:::2:head 2024-09-11T08:03:15.536 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: FIRST> seek to MIN oid=1 2024-09-11T08:03:15.536 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : oid=1 cursor=9:02547ec2:::1:head 2024-09-11T08:03:15.536 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:02547ec2:::1:head 2024-09-11T08:03:15.536 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:02547ec2:::1:head -> 1 2024-09-11T08:03:15.536 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : oid=10 cursor=9:52ea6a34:::10:head 2024-09-11T08:03:15.537 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:52ea6a34:::10:head 2024-09-11T08:03:15.537 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:52ea6a34:::10:head -> 10 2024-09-11T08:03:15.537 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : oid=13 cursor=9:566253c9:::13:head 2024-09-11T08:03:15.537 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:566253c9:::13:head 2024-09-11T08:03:15.537 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:566253c9:::13:head -> 13 2024-09-11T08:03:15.537 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : oid=7 cursor=9:5c6b0b28:::7:head 2024-09-11T08:03:15.537 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:5c6b0b28:::7:head 2024-09-11T08:03:15.537 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:5c6b0b28:::7:head -> 7 2024-09-11T08:03:15.537 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : oid=14 cursor=9:62a1935d:::14:head 2024-09-11T08:03:15.596 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:62a1935d:::14:head 2024-09-11T08:03:15.596 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:62a1935d:::14:head -> 14 2024-09-11T08:03:15.596 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : oid=0 cursor=9:6cac518f:::0:head 2024-09-11T08:03:15.596 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:6cac518f:::0:head 2024-09-11T08:03:15.596 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:6cac518f:::0:head -> 0 2024-09-11T08:03:15.596 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : oid=15 cursor=9:863748b0:::15:head 2024-09-11T08:03:15.596 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:863748b0:::15:head 2024-09-11T08:03:15.596 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:863748b0:::15:head -> 15 2024-09-11T08:03:15.596 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : oid=11 cursor=9:89d3ae78:::11:head 2024-09-11T08:03:15.596 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:89d3ae78:::11:head 2024-09-11T08:03:15.596 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:89d3ae78:::11:head -> 11 2024-09-11T08:03:15.596 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : oid=5 cursor=9:b29083e3:::5:head 2024-09-11T08:03:15.597 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:b29083e3:::5:head 2024-09-11T08:03:15.597 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:b29083e3:::5:head -> 5 2024-09-11T08:03:15.597 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : oid=8 cursor=9:bd63b0f1:::8:head 2024-09-11T08:03:15.597 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:bd63b0f1:::8:head 2024-09-11T08:03:15.597 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:bd63b0f1:::8:head -> 8 2024-09-11T08:03:15.597 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : oid=6 cursor=9:c4fdafeb:::6:head 2024-09-11T08:03:15.597 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:c4fdafeb:::6:head 2024-09-11T08:03:15.597 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:c4fdafeb:::6:head -> 6 2024-09-11T08:03:15.597 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : oid=3 cursor=9:cfc208b3:::3:head 2024-09-11T08:03:15.597 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:cfc208b3:::3:head 2024-09-11T08:03:15.597 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:cfc208b3:::3:head -> 3 2024-09-11T08:03:15.597 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : oid=4 cursor=9:d83876eb:::4:head 2024-09-11T08:03:15.598 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:d83876eb:::4:head 2024-09-11T08:03:15.598 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:d83876eb:::4:head -> 4 2024-09-11T08:03:15.598 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : oid=12 cursor=9:de5d7c5f:::12:head 2024-09-11T08:03:15.598 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:de5d7c5f:::12:head 2024-09-11T08:03:15.598 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:de5d7c5f:::12:head -> 12 2024-09-11T08:03:15.598 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : oid=9 cursor=9:e960b815:::9:head 2024-09-11T08:03:15.598 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:e960b815:::9:head 2024-09-11T08:03:15.598 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:e960b815:::9:head -> 9 2024-09-11T08:03:15.598 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : oid=2 cursor=9:f905c69b:::2:head 2024-09-11T08:03:15.598 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:f905c69b:::2:head 2024-09-11T08:03:15.598 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:f905c69b:::2:head -> 2 2024-09-11T08:03:15.598 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:f905c69b:::2:head 2024-09-11T08:03:15.599 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : cursor()=9:f905c69b:::2:head expected=9:f905c69b:::2:head 2024-09-11T08:03:15.599 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:f905c69b:::2:head -> 2 2024-09-11T08:03:15.599 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : entry=2 expected=2 2024-09-11T08:03:15.599 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:e960b815:::9:head 2024-09-11T08:03:15.599 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : cursor()=9:e960b815:::9:head expected=9:e960b815:::9:head 2024-09-11T08:03:15.599 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:e960b815:::9:head -> 9 2024-09-11T08:03:15.599 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : entry=9 expected=9 2024-09-11T08:03:15.599 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:de5d7c5f:::12:head 2024-09-11T08:03:15.599 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : cursor()=9:de5d7c5f:::12:head expected=9:de5d7c5f:::12:head 2024-09-11T08:03:15.599 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:de5d7c5f:::12:head -> 12 2024-09-11T08:03:15.599 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : entry=12 expected=12 2024-09-11T08:03:15.599 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:d83876eb:::4:head 2024-09-11T08:03:15.599 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : cursor()=9:d83876eb:::4:head expected=9:d83876eb:::4:head 2024-09-11T08:03:15.599 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:d83876eb:::4:head -> 4 2024-09-11T08:03:15.600 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : entry=4 expected=4 2024-09-11T08:03:15.600 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:cfc208b3:::3:head 2024-09-11T08:03:15.600 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : cursor()=9:cfc208b3:::3:head expected=9:cfc208b3:::3:head 2024-09-11T08:03:15.600 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:cfc208b3:::3:head -> 3 2024-09-11T08:03:15.600 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : entry=3 expected=3 2024-09-11T08:03:15.600 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:89d3ae78:::11:head 2024-09-11T08:03:15.600 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : cursor()=9:89d3ae78:::11:head expected=9:89d3ae78:::11:head 2024-09-11T08:03:15.600 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:89d3ae78:::11:head -> 11 2024-09-11T08:03:15.600 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : entry=11 expected=11 2024-09-11T08:03:15.600 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:02547ec2:::1:head 2024-09-11T08:03:15.600 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : cursor()=9:02547ec2:::1:head expected=9:02547ec2:::1:head 2024-09-11T08:03:15.600 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:02547ec2:::1:head -> 1 2024-09-11T08:03:15.600 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : entry=1 expected=1 2024-09-11T08:03:15.600 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:c4fdafeb:::6:head 2024-09-11T08:03:15.600 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : cursor()=9:c4fdafeb:::6:head expected=9:c4fdafeb:::6:head 2024-09-11T08:03:15.601 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:c4fdafeb:::6:head -> 6 2024-09-11T08:03:15.601 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : entry=6 expected=6 2024-09-11T08:03:15.601 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:bd63b0f1:::8:head 2024-09-11T08:03:15.739 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: list_parallel: process_1_[81154]: starting. 2024-09-11T08:03:15.739 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_1_[81154]: creating pool ceph_test_rados_list_parallel.smithi102-81119 2024-09-11T08:03:15.739 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_1_[81154]: created object 0... 2024-09-11T08:03:15.739 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_1_[81154]: created object 25... 2024-09-11T08:03:15.739 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_1_[81154]: created object 49... 2024-09-11T08:03:15.739 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_1_[81154]: finishing. 2024-09-11T08:03:15.739 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_1_[81154]: shutting down. 2024-09-11T08:03:15.739 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_2_[81155]: starting. 2024-09-11T08:03:15.739 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_2_[81155]: listing objects. 2024-09-11T08:03:15.739 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_2_[81155]: listed object 0... 2024-09-11T08:03:15.739 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_2_[81155]: listed object 25... 2024-09-11T08:03:15.739 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_2_[81155]: saw 50 objects 2024-09-11T08:03:15.740 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_2_[81155]: shutting down. 2024-09-11T08:03:15.740 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.740 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_3_[81859]: starting. 2024-09-11T08:03:15.740 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_3_[81859]: creating pool ceph_test_rados_list_parallel.smithi102-81119 2024-09-11T08:03:15.740 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_3_[81859]: created object 0... 2024-09-11T08:03:15.740 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_3_[81859]: created object 25... 2024-09-11T08:03:15.740 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_3_[81859]: created object 49... 2024-09-11T08:03:15.740 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_3_[81859]: finishing. 2024-09-11T08:03:15.740 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_3_[81859]: shutting down. 2024-09-11T08:03:15.740 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.740 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_4_[81860]: starting. 2024-09-11T08:03:15.740 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_4_[81860]: listing objects. 2024-09-11T08:03:15.740 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_4_[81860]: listed object 0... 2024-09-11T08:03:15.741 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_4_[81860]: listed object 25... 2024-09-11T08:03:15.741 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_4_[81860]: saw 43 objects 2024-09-11T08:03:15.741 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_4_[81860]: shutting down. 2024-09-11T08:03:15.741 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.741 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_5_[81861]: starting. 2024-09-11T08:03:15.741 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_5_[81861]: removed 25 objects... 2024-09-11T08:03:15.741 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_5_[81861]: removed half of the objects 2024-09-11T08:03:15.741 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_5_[81861]: removed 50 objects... 2024-09-11T08:03:15.741 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_5_[81861]: removed 50 objects 2024-09-11T08:03:15.741 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_5_[81861]: shutting down. 2024-09-11T08:03:15.741 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.741 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.741 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_6_[81907]: starting. 2024-09-11T08:03:15.742 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_6_[81907]: creating pool ceph_test_rados_list_parallel.smithi102-81119 2024-09-11T08:03:15.742 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_6_[81907]: created object 0... 2024-09-11T08:03:15.742 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_6_[81907]: created object 25... 2024-09-11T08:03:15.742 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_6_[81907]: created object 49... 2024-09-11T08:03:15.742 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_6_[81907]: finishing. 2024-09-11T08:03:15.742 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_6_[81907]: shutting down. 2024-09-11T08:03:15.742 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.742 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.742 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_7_[81908]: starting. 2024-09-11T08:03:15.742 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_7_[81908]: listing objects. 2024-09-11T08:03:15.742 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_7_[81908]: listed object 0... 2024-09-11T08:03:15.742 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_7_[81908]: listed object 25... 2024-09-11T08:03:15.742 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_7_[81908]: listed object 50... 2024-09-11T08:03:15.742 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_7_[81908]: saw 56 objects 2024-09-11T08:03:15.743 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_7_[81908]: shutting down. 2024-09-11T08:03:15.743 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.743 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.743 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_8_[81909]: starting. 2024-09-11T08:03:15.743 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_8_[81909]: added 25 objects... 2024-09-11T08:03:15.743 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_8_[81909]: added half of the objects 2024-09-11T08:03:15.743 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_8_[81909]: added 50 objects... 2024-09-11T08:03:15.743 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_8_[81909]: added 50 objects 2024-09-11T08:03:15.743 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_8_[81909]: shutting down. 2024-09-11T08:03:15.743 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.743 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.921 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.921 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_9_[81987]: starting. 2024-09-11T08:03:15.921 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_9_[81987]: creating pool ceph_test_rados_list_parallel.smithi102-81119 2024-09-11T08:03:15.921 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_9_[81987]: created object 0... 2024-09-11T08:03:15.921 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_9_[81987]: created object 25... 2024-09-11T08:03:15.921 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_9_[81987]: created object 49... 2024-09-11T08:03:15.921 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_9_[81987]: finishing. 2024-09-11T08:03:15.921 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_9_[81987]: shutting down. 2024-09-11T08:03:15.921 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.921 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.921 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.922 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_10_[81988]: starting. 2024-09-11T08:03:15.922 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_10_[81988]: listing objects. 2024-09-11T08:03:15.922 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_10_[81988]: listed object 0... 2024-09-11T08:03:15.922 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_10_[81988]: listed object 25... 2024-09-11T08:03:15.922 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_10_[81988]: listed object 50... 2024-09-11T08:03:15.922 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_10_[81988]: listed object 75... 2024-09-11T08:03:15.922 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_10_[81988]: saw 98 objects 2024-09-11T08:03:15.922 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_10_[81988]: shutting down. 2024-09-11T08:03:15.922 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.922 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.922 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.922 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_13_[81991]: starting. 2024-09-11T08:03:15.922 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_13_[81991]: removed 25 objects... 2024-09-11T08:03:15.923 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_13_[81991]: removed half of the objects 2024-09-11T08:03:15.923 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_13_[81991]: removed 50 objects... 2024-09-11T08:03:15.923 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_13_[81991]: removed 50 objects 2024-09-11T08:03:15.923 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_13_[81991]: shutting down. 2024-09-11T08:03:15.923 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.923 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.923 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.923 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_11_[81989]: starting. 2024-09-11T08:03:15.923 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_11_[81989]: added 25 objects... 2024-09-11T08:03:15.923 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_11_[81989]: added half of the objects 2024-09-11T08:03:15.923 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_11_[81989]: added 50 objects... 2024-09-11T08:03:15.923 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_11_[81989]: added 50 objects 2024-09-11T08:03:15.923 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_11_[81989]: shutting down. 2024-09-11T08:03:15.923 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.924 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.924 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.924 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_12_[81990]: starting. 2024-09-11T08:03:15.924 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_12_[81990]: added 25 objects... 2024-09-11T08:03:15.924 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_12_[81990]: added half of the objects 2024-09-11T08:03:15.924 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_12_[81990]: added 50 objects... 2024-09-11T08:03:15.924 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_12_[81990]: added 50 objects 2024-09-11T08:03:15.924 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_12_[81990]: shutting down. 2024-09-11T08:03:15.924 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.924 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.924 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.924 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.924 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_14_[82067]: starting. 2024-09-11T08:03:15.925 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_14_[82067]: creating pool ceph_test_rados_list_parallel.smithi102-81119 2024-09-11T08:03:15.925 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_14_[82067]: created object 0... 2024-09-11T08:03:15.925 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_14_[82067]: created object 25... 2024-09-11T08:03:15.925 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_14_[82067]: created object 49... 2024-09-11T08:03:15.925 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_14_[82067]: finishing. 2024-09-11T08:03:15.925 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_14_[82067]: shutting down. 2024-09-11T08:03:15.925 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.925 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.925 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.925 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:15.925 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_15_[82068]: starting. 2024-09-11T08:03:15.925 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_15_[82068]: listing objects. 2024-09-11T08:03:15.925 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_15_[82068]: listed object 0... 2024-09-11T08:03:16.557 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: from='client.24730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.557 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: from='client.24761 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi102-80885-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.557 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: from='client.24739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi102-80667-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.557 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: from='client.24695 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi102-80429-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.557 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: from='client.24742 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi102-80709-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.557 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: from='client.24830 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi102-81150-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.557 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: from='client.24685 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi102-80372-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.557 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: from='client.24692 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi102-80411-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.557 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: from='client.24688 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi102-80395-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.558 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: from='client.24709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi102-80461-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.558 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: from='client.24733 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi102-80697-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.558 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.558 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: from='client.24757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi102-80756-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.558 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: from='client.24814 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81109-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.558 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: from='client.24754 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi102-80768-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.558 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: from='client.24790 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.558 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: from='client.24844 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80973-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.558 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: from='client.24865 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80948-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.558 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: from='client.24763 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:16.558 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T08:03:16.558 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/929970514' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi102-80785-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:16.558 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi102-80785-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:16.559 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2501640046' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-11T08:03:16.559 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: from='client.24730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: from='client.24761 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi102-80885-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: from='client.24739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi102-80667-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: from='client.24695 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi102-80429-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: from='client.24742 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi102-80709-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: from='client.24830 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi102-81150-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: from='client.24685 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi102-80372-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: from='client.24692 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi102-80411-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: from='client.24688 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi102-80395-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: from='client.24709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi102-80461-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: from='client.24733 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi102-80697-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: from='client.24757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi102-80756-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: from='client.24814 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81109-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: from='client.24754 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi102-80768-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: from='client.24790 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: from='client.24844 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80973-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: from='client.24730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: from='client.24761 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi102-80885-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: from='client.24739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi102-80667-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: from='client.24695 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi102-80429-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: from='client.24742 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi102-80709-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: from='client.24830 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi102-81150-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: from='client.24685 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi102-80372-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: from='client.24692 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi102-80411-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: from='client.24688 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi102-80395-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: from='client.24709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi102-80461-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: from='client.24733 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi102-80697-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: from='client.24757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi102-80756-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: from='client.24814 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81109-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: from='client.24754 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi102-80768-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: from='client.24790 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: from='client.24844 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80973-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: from='client.24865 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80948-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: from='client.24763 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:16.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T08:03:16.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/929970514' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi102-80785-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:16.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi102-80785-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:16.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2501640046' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-11T08:03:16.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:16.650 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: from='client.24865 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80948-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:16.650 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: from='client.24763 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:16.650 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T08:03:16.650 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/929970514' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi102-80785-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:16.650 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi102-80785-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:16.650 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2501640046' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-11T08:03:16.650 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:16.960 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_15_[82068]: listed object 25... 2024-09-11T08:03:16.961 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_15_[82068]: listed object 50... 2024-09-11T08:03:16.961 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_15_[82068]: listed object 75... 2024-09-11T08:03:16.961 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_15_[82068]: listed object 100... 2024-09-11T08:03:16.961 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_15_[82068]: listed object 125... 2024-09-11T08:03:16.961 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_15_[82068]: saw 150 objects 2024-09-11T08:03:16.961 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_15_[82068]: shutting down. 2024-09-11T08:03:16.961 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:16.961 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:16.961 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:16.961 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:16.961 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_16_[82069]: starting. 2024-09-11T08:03:16.961 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_16_[82069]: added 25 objects... 2024-09-11T08:03:16.961 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_16_[82069]: added half of the objects 2024-09-11T08:03:16.962 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_16_[82069]: added 50 objects... 2024-09-11T08:03:16.962 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_16_[82069]: added 50 objects 2024-09-11T08:03:16.962 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: process_16_[82069]: shutting down. 2024-09-11T08:03:16.962 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:16.962 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:16.962 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:16.962 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:16.962 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******************************* 2024-09-11T08:03:16.962 INFO:tasks.workunit.client.0.smithi102.stdout: list_parallel: ******* SUCCESS ********** 2024-09-11T08:03:16.963 INFO:tasks.workunit.client.0.smithi102.stdout: api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2024-09-11T08:03:16.963 INFO:tasks.workunit.client.0.smithi102.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2024-09-11T08:03:16.963 INFO:tasks.workunit.client.0.smithi102.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2024-09-11T08:03:16.963 INFO:tasks.workunit.client.0.smithi102.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2024-09-11T08:03:16.963 INFO:tasks.workunit.client.0.smithi102.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (8582 ms) 2024-09-11T08:03:16.963 INFO:tasks.workunit.client.0.smithi102.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (8582 ms total) 2024-09-11T08:03:16.963 INFO:tasks.workunit.client.0.smithi102.stdout: api_cls_remote_reads: 2024-09-11T08:03:16.963 INFO:tasks.workunit.client.0.smithi102.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2024-09-11T08:03:16.963 INFO:tasks.workunit.client.0.smithi102.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (8582 ms total) 2024-09-11T08:03:16.963 INFO:tasks.workunit.client.0.smithi102.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2024-09-11T08:03:16.969 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: Running main() from gmock_main.cc 2024-09-11T08:03:16.969 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [==========] Running 16 tests from 1 test suite. 2024-09-11T08:03:16.970 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-09-11T08:03:16.970 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [----------] 16 tests from CReadOpsTest 2024-09-11T08:03:16.970 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-09-11T08:03:16.970 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-09-11T08:03:16.970 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-09-11T08:03:16.970 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (248 ms) 2024-09-11T08:03:16.970 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-09-11T08:03:16.970 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (5 ms) 2024-09-11T08:03:16.970 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-09-11T08:03:16.970 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (31 ms) 2024-09-11T08:03:16.970 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-09-11T08:03:16.970 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (35 ms) 2024-09-11T08:03:16.970 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-09-11T08:03:16.971 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (7 ms) 2024-09-11T08:03:16.971 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-09-11T08:03:16.971 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (7 ms) 2024-09-11T08:03:16.971 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-09-11T08:03:16.971 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (6 ms) 2024-09-11T08:03:16.971 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-09-11T08:03:16.971 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (10 ms) 2024-09-11T08:03:16.971 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-09-11T08:03:16.971 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (16 ms) 2024-09-11T08:03:16.971 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-09-11T08:03:16.972 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (13 ms) 2024-09-11T08:03:16.972 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-09-11T08:03:16.972 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (5 ms) 2024-09-11T08:03:16.972 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-09-11T08:03:16.972 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (14 ms) 2024-09-11T08:03:16.972 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-09-11T08:03:16.972 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (8 ms) 2024-09-11T08:03:16.972 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-09-11T08:03:16.972 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (8 ms) 2024-09-11T08:03:16.972 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-09-11T08:03:16.972 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (6 ms) 2024-09-11T08:03:16.972 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [----------] 16 tests from CReadOpsTest (419 ms total) 2024-09-11T08:03:16.972 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: 2024-09-11T08:03:16.972 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-09-11T08:03:16.973 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [==========] 16 tests from 1 test suite ran. (8570 ms total) 2024-09-11T08:03:16.973 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_read_operations: [ PASSED ] 16 tests. 2024-09-11T08:03:16.992 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd_pp: Running main() from gmock_main.cc 2024-09-11T08:03:16.992 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-09-11T08:03:16.992 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-09-11T08:03:16.992 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-09-11T08:03:16.992 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-09-11T08:03:16.992 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (43 ms) 2024-09-11T08:03:16.992 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-09-11T08:03:16.992 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (45 ms) 2024-09-11T08:03:16.993 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-09-11T08:03:16.993 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (8582 ms) 2024-09-11T08:03:16.993 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (8670 ms total) 2024-09-11T08:03:16.993 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd_pp: 2024-09-11T08:03:16.993 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-09-11T08:03:16.993 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (8670 ms total) 2024-09-11T08:03:16.993 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-09-11T08:03:17.057 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: Running main() from gmock_main.cc 2024-09-11T08:03:17.058 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-09-11T08:03:17.058 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: [----------] Global test environment set-up. 2024-09-11T08:03:17.058 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-09-11T08:03:17.058 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-09-11T08:03:17.058 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (52 ms) 2024-09-11T08:03:17.058 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-09-11T08:03:17.058 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (47 ms) 2024-09-11T08:03:17.058 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-09-11T08:03:17.058 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (8554 ms) 2024-09-11T08:03:17.058 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-09-11T08:03:17.058 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:14.944314+0000 mon.a [INF] from='client.24730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:17.058 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:14.944392+0000 mon.a [INF] from='client.24761 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi102-80885-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:17.059 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:14.944446+0000 mon.a [INF] from='client.24739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi102-80667-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:17.059 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:14.944496+0000 mon.a [INF] from='client.24695 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi102-80429-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:17.059 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:14.944591+0000 mon.a [INF] from='client.24742 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi102-80709-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:17.059 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:14.944646+0000 mon.a [INF] from='client.24830 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi102-81150-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:17.059 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:14.944706+0000 mon.a [INF] from='client.24685 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi102-80372-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:17.059 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:14.944758+0000 mon.a [INF] from='client.24692 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi102-80411-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:17.059 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:14.944824+0000 mon.a [INF] from='client.24688 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi102-80395-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:17.059 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:14.944903+0000 mon.a [INF] from='client.24709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi102-80461-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:17.059 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:14.944984+0000 mon.a [INF] from='client.24733 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi102-80697-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:17.059 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:14.945194+0000 mon.a [INF] from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:17.779 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:14.945330+0000 mon.a [INF] from='client.24757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi102-80756-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:17.780 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:14.945390+0000 mon.a [INF] from='client.24814 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81109-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:17.780 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:14.945503+0000 mon.a [INF] from='client.24754 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi102-80768-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:17.780 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:14.945606+0000 mon.a [INF] from='client.24790 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:17.780 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:14.945665+0000 mon.a [INF] from='client.24844 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80973-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:17.780 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:14.945727+0000 mon.a [INF] from='client.24865 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80948-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:17.780 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:14.945835+0000 mon.a [INF] from='client.24763 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:17.780 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:14.948080+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/929970514' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi102-80785-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:17.780 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:14.968627+0000 mon.a [INF] from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi102-80785-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:17.780 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:16.231606+0000 mon.a [INF] from='client.? v1:172.21.15.102:0/1614749820' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi102-80356-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:17.781 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:16.231834+0000 mon.a [INF] from='client.? v1:172.21.15.102:0/1989677394' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi102-80338-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:17.781 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:16.955312+0000 mon.a [INF] from='client.24763 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi102-80785-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]': finished 2024-09-11T08:03:17.781 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:16.955394+0000 mon.a [INF] from='client.? v1:172.21.15.102:0/1614749820' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi102-80356-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:17.781 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:16.955442+0000 mon.a [INF] from='client.? v1:172.21.15.102:0/1989677394' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi102-80338-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:18.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[29849]: osdmap e68: 8 total, 8 up, 8 in 2024-09-11T08:03:18.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1614749820' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi102-80356-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1989677394' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi102-80338-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[29849]: pgmap v71: 932 pgs: 32 creating+peering, 96 unknown, 804 active+clean; 632 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 23 KiB/s rd, 59 KiB/s wr, 93 op/s 2024-09-11T08:03:18.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:18.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[29849]: from='client.24763 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi102-80785-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]': finished 2024-09-11T08:03:18.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1614749820' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi102-80356-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:18.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1989677394' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi102-80338-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:18.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[29849]: osdmap e69: 8 total, 8 up, 8 in 2024-09-11T08:03:18.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2259835227' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1074126277' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[29849]: from='client.25012 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[29849]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[41371]: osdmap e68: 8 total, 8 up, 8 in 2024-09-11T08:03:18.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1614749820' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi102-80356-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1989677394' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi102-80338-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[41371]: pgmap v71: 932 pgs: 32 creating+peering, 96 unknown, 804 active+clean; 632 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 23 KiB/s rd, 59 KiB/s wr, 93 op/s 2024-09-11T08:03:18.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:18.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[41371]: from='client.24763 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi102-80785-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]': finished 2024-09-11T08:03:18.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1614749820' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi102-80356-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:18.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1989677394' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi102-80338-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:18.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[41371]: osdmap e69: 8 total, 8 up, 8 in 2024-09-11T08:03:18.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2259835227' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1074126277' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[41371]: from='client.25012 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:17 smithi102 ceph-mon[41371]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:17 smithi146 ceph-mon[33158]: osdmap e68: 8 total, 8 up, 8 in 2024-09-11T08:03:18.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1614749820' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi102-80356-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1989677394' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi102-80338-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:17 smithi146 ceph-mon[33158]: pgmap v71: 932 pgs: 32 creating+peering, 96 unknown, 804 active+clean; 632 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 23 KiB/s rd, 59 KiB/s wr, 93 op/s 2024-09-11T08:03:18.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:18.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:17 smithi146 ceph-mon[33158]: from='client.24763 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi102-80785-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]': finished 2024-09-11T08:03:18.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1614749820' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi102-80356-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:18.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1989677394' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi102-80338-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:18.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:17 smithi146 ceph-mon[33158]: osdmap e69: 8 total, 8 up, 8 in 2024-09-11T08:03:18.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2259835227' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1074126277' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:17 smithi146 ceph-mon[33158]: from='client.25012 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:17 smithi146 ceph-mon[33158]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.782 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:16.957713+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/2259835227' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.782 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:16.959008+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/1074126277' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.782 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:16.966396+0000 mon.a [INF] from='client.25012 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.782 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:16.968722+0000 mon.a [INF] from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.783 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:17.039652+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/2807324871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:18.783 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:17.051237+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/3089827920' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:18.783 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:17.053356+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/1738035696' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-11T08:03:18.783 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:17.055021+0000 client.admin [INF] onexx 2024-09-11T08:03:18.783 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:17.057854+0000 mon.c [INF] from='client.? v1:172.21.15.102:0/3846439703' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:18.783 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:17.075429+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/637746560' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:18.783 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:17.530966+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.0"}]: dispatch 2024-09-11T08:03:18.783 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:17.531566+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.1"}]: dispatch 2024-09-11T08:03:18.783 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:17.531957+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.2"}]: dispatch 2024-09-11T08:03:18.783 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:17.532305+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.3"}]: dispatch 2024-09-11T08:03:18.783 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:17.532656+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.4"}]: dispatch 2024-09-11T08:03:18.784 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:17.533019+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.5"}]: dispatch 2024-09-11T08:03:18.784 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:17.533375+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.6"}]: dispatch 2024-09-11T08:03:18.784 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:17.533712+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.7"}]: dispatch 2024-09-11T08:03:18.784 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:17.534055+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.8"}]: dispatch 2024-09-11T08:03:18.784 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:17.534435+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.9"}]: dispatch 2024-09-11T08:03:18.784 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:17.540654+0000 mon.a [INF] from='client.25124 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:18.784 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:17.543255+0000 mon.c [INF] from='client.? v1:172.21.15.102:0/3846439703' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:18.784 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:17.782818+0000 mon.a [INF] from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:18.784 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:17.783526+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/2807324871' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:18.784 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:18.021498+0000 mon.a [INF] from='client.25159 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:18.784 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:18.021690+0000 mon.a [INF] from='client.25153 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-11T08:03:18.785 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:18.028484+0000 mon.a [INF] from='client.25012 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:18.785 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:18.028609+0000 mon.a [INF] from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:18.785 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:18.031661+0000 mon.c [INF] from='client.? v1:172.21.15.102:0/2080632986' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.785 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:18.032169+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/3089827920' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:18.785 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:18.032681+0000 mon.a [INF] from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:18.785 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:18.045988+0000 mon.a [INF] from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:18.785 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:18.047171+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/637746560' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:18.785 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:18.047273+0000 mon.a [INF] from='client.? v1:172.21.15.102:0/1614749820' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi102-80356-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:18.785 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:18.047660+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/2807324871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:18.785 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:18.048173+0000 mon.a [INF] from='client.25159 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:19.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2807324871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:19.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3089827920' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:19.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1738035696' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-11T08:03:19.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: onexx 2024-09-11T08:03:19.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3846439703' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:19.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/637746560' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:19.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.0"}]: dispatch 2024-09-11T08:03:19.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.0"}]: dispatch 2024-09-11T08:03:19.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.1"}]: dispatch 2024-09-11T08:03:19.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.1"}]: dispatch 2024-09-11T08:03:19.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.2"}]: dispatch 2024-09-11T08:03:19.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.2"}]: dispatch 2024-09-11T08:03:19.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.3"}]: dispatch 2024-09-11T08:03:19.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.3"}]: dispatch 2024-09-11T08:03:19.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.4"}]: dispatch 2024-09-11T08:03:19.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.4"}]: dispatch 2024-09-11T08:03:19.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.5"}]: dispatch 2024-09-11T08:03:19.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.5"}]: dispatch 2024-09-11T08:03:19.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.6"}]: dispatch 2024-09-11T08:03:19.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.6"}]: dispatch 2024-09-11T08:03:19.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.7"}]: dispatch 2024-09-11T08:03:19.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.7"}]: dispatch 2024-09-11T08:03:19.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.8"}]: dispatch 2024-09-11T08:03:19.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.8"}]: dispatch 2024-09-11T08:03:19.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2807324871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:19.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3089827920' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:19.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1738035696' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-11T08:03:19.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: onexx 2024-09-11T08:03:19.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3846439703' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:19.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/637746560' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:19.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.0"}]: dispatch 2024-09-11T08:03:19.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.0"}]: dispatch 2024-09-11T08:03:19.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.1"}]: dispatch 2024-09-11T08:03:19.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.1"}]: dispatch 2024-09-11T08:03:19.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.2"}]: dispatch 2024-09-11T08:03:19.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.2"}]: dispatch 2024-09-11T08:03:19.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.3"}]: dispatch 2024-09-11T08:03:19.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.3"}]: dispatch 2024-09-11T08:03:19.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.4"}]: dispatch 2024-09-11T08:03:19.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.4"}]: dispatch 2024-09-11T08:03:19.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.5"}]: dispatch 2024-09-11T08:03:19.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.5"}]: dispatch 2024-09-11T08:03:19.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.6"}]: dispatch 2024-09-11T08:03:19.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.6"}]: dispatch 2024-09-11T08:03:19.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.7"}]: dispatch 2024-09-11T08:03:19.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.7"}]: dispatch 2024-09-11T08:03:19.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.8"}]: dispatch 2024-09-11T08:03:19.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.8"}]: dispatch 2024-09-11T08:03:19.151 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.9"}]: dispatch 2024-09-11T08:03:19.151 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.9"}]: dispatch 2024-09-11T08:03:19.151 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.25124 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:19.151 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3846439703' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:19.151 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:19.151 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:19.151 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2807324871' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:19.151 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: 22.2 deep-scrub starts 2024-09-11T08:03:19.151 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: 22.2 deep-scrub ok 2024-09-11T08:03:19.152 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.25159 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:19.152 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.25153 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-11T08:03:19.152 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.25012 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:19.152 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:19.152 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: osdmap e70: 8 total, 8 up, 8 in 2024-09-11T08:03:19.152 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2080632986' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:19.152 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3089827920' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:19.152 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:19.152 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:19.152 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/637746560' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:19.152 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1614749820' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi102-80356-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:19.152 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2807324871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.153 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.25159 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:19.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.9"}]: dispatch 2024-09-11T08:03:19.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.9"}]: dispatch 2024-09-11T08:03:19.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.25124 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:19.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3846439703' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:19.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:19.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:19.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2807324871' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:19.153 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: 22.2 deep-scrub starts 2024-09-11T08:03:19.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: 22.2 deep-scrub ok 2024-09-11T08:03:19.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.25159 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:19.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.25153 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-11T08:03:19.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.25012 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:19.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:19.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: osdmap e70: 8 total, 8 up, 8 in 2024-09-11T08:03:19.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2080632986' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:19.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3089827920' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:19.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:19.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:19.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/637746560' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:19.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1614749820' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi102-80356-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:19.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2807324871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.154 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.25159 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:19.155 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3089827920' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi102-80885-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.155 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: 22.6 deep-scrub starts 2024-09-11T08:03:19.155 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:19.155 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.25124 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:19.155 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.155 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/637746560' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.155 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3846439703' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi102-80372-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.155 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.25159 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi102-80885-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.155 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.25121 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:19.156 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.25124 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi102-80372-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.156 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.156 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:19.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3089827920' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi102-80885-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: 22.6 deep-scrub starts 2024-09-11T08:03:19.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:19.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.25124 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:19.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/637746560' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3846439703' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi102-80372-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.156 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.25159 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi102-80885-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.25121 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:19.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.25124 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi102-80372-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.157 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:19.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2807324871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:19.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3089827920' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:19.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1738035696' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-11T08:03:19.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: onexx 2024-09-11T08:03:19.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3846439703' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:19.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/637746560' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:19.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.0"}]: dispatch 2024-09-11T08:03:19.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.0"}]: dispatch 2024-09-11T08:03:19.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.1"}]: dispatch 2024-09-11T08:03:19.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.1"}]: dispatch 2024-09-11T08:03:19.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.2"}]: dispatch 2024-09-11T08:03:19.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.2"}]: dispatch 2024-09-11T08:03:19.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.3"}]: dispatch 2024-09-11T08:03:19.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.3"}]: dispatch 2024-09-11T08:03:19.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.4"}]: dispatch 2024-09-11T08:03:19.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.4"}]: dispatch 2024-09-11T08:03:19.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.5"}]: dispatch 2024-09-11T08:03:19.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.5"}]: dispatch 2024-09-11T08:03:19.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.6"}]: dispatch 2024-09-11T08:03:19.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.6"}]: dispatch 2024-09-11T08:03:19.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.7"}]: dispatch 2024-09-11T08:03:19.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.7"}]: dispatch 2024-09-11T08:03:19.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.8"}]: dispatch 2024-09-11T08:03:19.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.8"}]: dispatch 2024-09-11T08:03:19.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.9"}]: dispatch 2024-09-11T08:03:19.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.9"}]: dispatch 2024-09-11T08:03:19.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.25124 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:19.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3846439703' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:19.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:19.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:19.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2807324871' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:19.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: 22.2 deep-scrub starts 2024-09-11T08:03:19.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: 22.2 deep-scrub ok 2024-09-11T08:03:19.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.25159 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:19.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.25153 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-11T08:03:19.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.25012 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:19.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:19.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: osdmap e70: 8 total, 8 up, 8 in 2024-09-11T08:03:19.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2080632986' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:19.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3089827920' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:19.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:19.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:19.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/637746560' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:19.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1614749820' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi102-80356-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:19.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2807324871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.25159 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:19.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3089827920' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi102-80885-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: 22.6 deep-scrub starts 2024-09-11T08:03:19.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:19.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.25124 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:19.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/637746560' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3846439703' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi102-80372-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.25159 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi102-80885-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.25121 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:19.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.25124 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi102-80372-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.189 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:19.189 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:19.782 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:03:20.046 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:03:19] "GET /metrics HTTP/1.1" 200 276515 "" "Prometheus/2.43.0" 2024-09-11T08:03:20.047 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: pgmap v74: 716 pgs: 32 creating+peering, 264 unknown, 420 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 10 KiB/s wr, 27 op/s 2024-09-11T08:03:20.047 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: 22.8 deep-scrub starts 2024-09-11T08:03:20.047 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: 22.8 deep-scrub ok 2024-09-11T08:03:20.047 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: 22.0 deep-scrub starts 2024-09-11T08:03:20.047 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: 22.0 deep-scrub ok 2024-09-11T08:03:20.047 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.25153 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-11T08:03:20.048 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1738035696' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-11T08:03:20.048 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: twoxx 2024-09-11T08:03:20.048 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.25153 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-11T08:03:20.048 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: 22.7 deep-scrub starts 2024-09-11T08:03:20.048 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: 22.1 deep-scrub starts 2024-09-11T08:03:20.048 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: 22.7 deep-scrub ok 2024-09-11T08:03:20.048 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: 22.1 deep-scrub ok 2024-09-11T08:03:20.048 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: 22.9 deep-scrub starts 2024-09-11T08:03:20.048 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: 22.9 deep-scrub ok 2024-09-11T08:03:20.048 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1614749820' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi102-80356-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:20.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:20.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.25159 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi102-80885-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:20.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.25121 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:20.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.25124 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi102-80372-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:20.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:20.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: osdmap e71: 8 total, 8 up, 8 in 2024-09-11T08:03:20.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3846439703' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi102-80372-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:20.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1264347479' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi102-80338-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:20.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.25124 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi102-80372-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:20.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1614749820' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi102-80356-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-11T08:03:20.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.25148 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi102-80338-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:20.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2259835227' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:20.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2807324871' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi102-80756-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:20.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3089827920' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi102-80885-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:20.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/637746560' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi102-80395-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:20.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.25012 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:20.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi102-80756-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:20.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.25159 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi102-80885-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:20.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi102-80395-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:20.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: 22.6 deep-scrub starts 2024-09-11T08:03:20.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3051776469' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:20.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.25195 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:20.050 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1722176585' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:20.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.25192 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:20.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3051776469' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:20.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.25195 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:20.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1722176585' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:20.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.25192 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:20.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3051776469' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi102-80461-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:20.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.25195 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi102-80461-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:20.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1722176585' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:20.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.25192 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:20.051 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:20.052 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: pgmap v74: 716 pgs: 32 creating+peering, 264 unknown, 420 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 10 KiB/s wr, 27 op/s 2024-09-11T08:03:20.052 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: 22.8 deep-scrub starts 2024-09-11T08:03:20.052 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: 22.8 deep-scrub ok 2024-09-11T08:03:20.052 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: 22.0 deep-scrub starts 2024-09-11T08:03:20.052 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: 22.0 deep-scrub ok 2024-09-11T08:03:20.052 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.25153 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-11T08:03:20.052 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1738035696' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-11T08:03:20.053 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: twoxx 2024-09-11T08:03:20.053 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.25153 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-11T08:03:20.053 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: 22.7 deep-scrub starts 2024-09-11T08:03:20.053 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: 22.1 deep-scrub starts 2024-09-11T08:03:20.053 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: 22.7 deep-scrub ok 2024-09-11T08:03:20.053 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: 22.1 deep-scrub ok 2024-09-11T08:03:20.053 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: 22.9 deep-scrub starts 2024-09-11T08:03:20.053 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: 22.9 deep-scrub ok 2024-09-11T08:03:20.053 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1614749820' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi102-80356-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:20.053 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:20.053 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.25159 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi102-80885-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:20.054 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.25121 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:20.054 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.25124 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi102-80372-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:20.054 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:20.054 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: osdmap e71: 8 total, 8 up, 8 in 2024-09-11T08:03:20.054 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3846439703' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi102-80372-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:20.054 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1264347479' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi102-80338-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:20.054 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.25124 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi102-80372-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:20.054 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1614749820' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi102-80356-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-11T08:03:20.054 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.25148 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi102-80338-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:20.054 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2259835227' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:20.054 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2807324871' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi102-80756-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:20.055 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3089827920' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi102-80885-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:20.055 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/637746560' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi102-80395-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:20.055 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.25012 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:20.055 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi102-80756-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:20.055 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.25159 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi102-80885-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:20.055 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi102-80395-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:20.055 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: 22.6 deep-scrub starts 2024-09-11T08:03:20.055 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3051776469' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:20.055 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.25195 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:20.055 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1722176585' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:20.055 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.25192 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:20.056 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3051776469' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:20.056 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.25195 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:20.056 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1722176585' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:20.056 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.25192 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:20.056 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3051776469' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi102-80461-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:20.056 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.25195 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi102-80461-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:20.056 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1722176585' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:20.056 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.25192 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:20.056 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:20.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: pgmap v74: 716 pgs: 32 creating+peering, 264 unknown, 420 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 10 KiB/s wr, 27 op/s 2024-09-11T08:03:20.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: 22.8 deep-scrub starts 2024-09-11T08:03:20.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: 22.8 deep-scrub ok 2024-09-11T08:03:20.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: 22.0 deep-scrub starts 2024-09-11T08:03:20.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: 22.0 deep-scrub ok 2024-09-11T08:03:20.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.25153 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-11T08:03:20.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1738035696' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-11T08:03:20.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: twoxx 2024-09-11T08:03:20.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.25153 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-11T08:03:20.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: 22.7 deep-scrub starts 2024-09-11T08:03:20.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: 22.1 deep-scrub starts 2024-09-11T08:03:20.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: 22.7 deep-scrub ok 2024-09-11T08:03:20.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: 22.1 deep-scrub ok 2024-09-11T08:03:20.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: 22.9 deep-scrub starts 2024-09-11T08:03:20.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: 22.9 deep-scrub ok 2024-09-11T08:03:20.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1614749820' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi102-80356-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:20.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:20.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.25159 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi102-80885-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:20.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.25121 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:20.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.25124 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi102-80372-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:20.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:20.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: osdmap e71: 8 total, 8 up, 8 in 2024-09-11T08:03:20.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3846439703' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi102-80372-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:20.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1264347479' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi102-80338-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:20.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.25124 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi102-80372-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:20.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1614749820' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi102-80356-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-11T08:03:20.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.25148 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi102-80338-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:20.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2259835227' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:20.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2807324871' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi102-80756-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:20.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3089827920' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi102-80885-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:20.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/637746560' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi102-80395-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:20.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.25012 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:20.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi102-80756-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:20.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.25159 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi102-80885-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:20.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi102-80395-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:20.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: 22.6 deep-scrub starts 2024-09-11T08:03:20.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3051776469' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:20.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.25195 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:20.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1722176585' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:20.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.25192 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:20.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3051776469' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:20.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.25195 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:20.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1722176585' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:20.188 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.25192 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:20.189 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3051776469' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi102-80461-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:20.189 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.25195 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi102-80461-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:20.189 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1722176585' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:20.189 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.25192 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:20.189 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:21.010 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: process_1_[81231]: starting. 2024-09-11T08:03:21.010 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: process_1_[81231]: creating pool ceph_test_rados_open_pools_parallel.smithi102-81196 2024-09-11T08:03:21.010 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: process_1_[81231]: created object 0... 2024-09-11T08:03:21.010 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: process_1_[81231]: created object 25... 2024-09-11T08:03:21.010 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: process_1_[81231]: created object 49... 2024-09-11T08:03:21.010 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: process_1_[81231]: finishing. 2024-09-11T08:03:21.010 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: process_1_[81231]: shutting down. 2024-09-11T08:03:21.010 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: process_2_[81232]: starting. 2024-09-11T08:03:21.010 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: process_2_[81232]: rados_pool_create. 2024-09-11T08:03:21.010 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: process_2_[81232]: rados_ioctx_create. 2024-09-11T08:03:21.011 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: process_2_[81232]: shutting down. 2024-09-11T08:03:21.011 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: ******************************* 2024-09-11T08:03:21.011 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: process_3_[82156]: starting. 2024-09-11T08:03:21.011 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: process_3_[82156]: creating pool ceph_test_rados_open_pools_parallel.smithi102-81196 2024-09-11T08:03:21.011 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: process_3_[82156]: created object 0... 2024-09-11T08:03:21.011 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: process_3_[82156]: created object 25... 2024-09-11T08:03:21.011 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: process_3_[82156]: created object 49... 2024-09-11T08:03:21.011 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: process_3_[82156]: finishing. 2024-09-11T08:03:21.011 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: process_3_[82156]: shutting down. 2024-09-11T08:03:21.011 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: ******************************* 2024-09-11T08:03:21.011 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: process_4_[82157]: starting. 2024-09-11T08:03:21.012 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: process_4_[82157]: rados_pool_create. 2024-09-11T08:03:21.012 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: process_4_[82157]: rados_ioctx_create. 2024-09-11T08:03:21.012 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: process_4_[82157]: shutting down. 2024-09-11T08:03:21.012 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: ******************************* 2024-09-11T08:03:21.012 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: ******************************* 2024-09-11T08:03:21.012 INFO:tasks.workunit.client.0.smithi102.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-09-11T08:03:21.012 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_1_[81222]: starting. 2024-09-11T08:03:21.012 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_1_[81222]: creating pool ceph_test_rados_delete_pools_parallel.smithi102-81189 2024-09-11T08:03:21.012 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_1_[81222]: created object 0... 2024-09-11T08:03:21.013 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_1_[81222]: created object 25... 2024-09-11T08:03:21.013 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_1_[81222]: created object 49... 2024-09-11T08:03:21.013 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_1_[81222]: finishing. 2024-09-11T08:03:21.013 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_1_[81222]: shutting down. 2024-09-11T08:03:21.013 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_2_[81223]: starting. 2024-09-11T08:03:21.013 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_2_[81223]: deleting pool ceph_test_rados_delete_pools_parallel.smithi102-81189 2024-09-11T08:03:21.013 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_2_[81223]: shutting down. 2024-09-11T08:03:21.013 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: ******************************* 2024-09-11T08:03:21.013 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_3_[82152]: starting. 2024-09-11T08:03:21.013 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_3_[82152]: creating pool ceph_test_rados_delete_pools_parallel.smithi102-81189 2024-09-11T08:03:21.013 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_3_[82152]: created object 0... 2024-09-11T08:03:21.013 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_3_[82152]: created object 25... 2024-09-11T08:03:21.013 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_3_[82152]: created object 49... 2024-09-11T08:03:21.014 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_3_[82152]: finishing. 2024-09-11T08:03:21.014 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_3_[82152]: shutting down. 2024-09-11T08:03:21.014 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: ******************************* 2024-09-11T08:03:21.014 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_5_[82154]: starting. 2024-09-11T08:03:21.014 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_5_[82154]: listing objects. 2024-09-11T08:03:21.014 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_5_[82154]: listed object 0... 2024-09-11T08:03:21.014 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_5_[82154]: listed object 25... 2024-09-11T08:03:21.014 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_5_[82154]: saw 50 objects 2024-09-11T08:03:21.014 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_5_[82154]: shutting down. 2024-09-11T08:03:21.014 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: ******************************* 2024-09-11T08:03:21.014 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_4_[82153]: starting. 2024-09-11T08:03:21.014 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_4_[82153]: deleting pool ceph_test_rados_delete_pools_parallel.smithi102-81189 2024-09-11T08:03:21.014 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: process_4_[82153]: shutting down. 2024-09-11T08:03:21.015 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: ******************************* 2024-09-11T08:03:21.015 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: ******************************* 2024-09-11T08:03:21.015 INFO:tasks.workunit.client.0.smithi102.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-09-11T08:03:21.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:21 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:03:21.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:21 smithi102 ceph-mon[41371]: from='client.25153 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-11T08:03:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:21 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:03:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:21 smithi102 ceph-mon[29849]: from='client.25153 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-11T08:03:21.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:21 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:03:21.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:21 smithi146 ceph-mon[33158]: from='client.25153 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-11T08:03:22.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:18.051570+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/3089827920' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi102-80885-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:22.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:18.312342+0000 mon.a [INF] from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:22.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:18.312741+0000 mon.a [INF] from='client.25124 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:22.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:18.313013+0000 mon.a [INF] from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:22.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:18.313069+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/637746560' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:22.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:18.313201+0000 mon.c [INF] from='client.? v1:172.21.15.102:0/3846439703' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi102-80372-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:22.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:18.313674+0000 mon.a [INF] from='client.25159 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi102-80885-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:22.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:18.313943+0000 mon.a [INF] from='client.25121 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:22.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:18.554196+0000 mon.a [INF] from='client.25124 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi102-80372-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:22.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:18.556227+0000 mon.a [INF] from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:22.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:19.785502+0000 mon.a [INF] from='client.25153 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-11T08:03:22.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:20.298689+0000 mon.a [INF] from='client.? v1:172.21.15.102:0/1614749820' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi102-80356-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-11T08:03:22.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:20.298765+0000 mon.a [INF] from='client.25148 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi102-80338-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:22.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:20.298864+0000 mon.a [INF] from='client.25012 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:22.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:20.299949+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/2259835227' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-4", "tierpool": "test-rados-api-smithi102-80667-4-cache"}]: dispatch 2024-09-11T08:03:22.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:21.004387+0000 mon.a [INF] from='client.25195 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi102-80461-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:22.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:21.004480+0000 mon.a [INF] from='client.25192 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:22.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:21.005234+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/3051776469' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi102-80461-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:22.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:21.005638+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/1722176585' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi102-80429-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:22.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:21.006436+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/4227672529' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:22.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:21.006555+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/929970514' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:22.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:21.272827+0000 mon.a [INF] from='client.25124 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi102-80372-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi102-80372-16"}]': finished 2024-09-11T08:03:22.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:21.272929+0000 mon.a [INF] from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi102-80756-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi102-80756-7"}]': finished 2024-09-11T08:03:22.017 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:21.272999+0000 mon.a [INF] from='client.25159 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi102-80885-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi102-80885-6"}]': finished 2024-09-11T08:03:22.017 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:21.273068+0000 mon.a [INF] from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi102-80395-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi102-80395-22"}]': finished 2024-09-11T08:03:22.017 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:21.274388+0000 mon.c [INF] from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:22.017 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:21.525138+0000 mon.a [INF] from='client.25166 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:22.017 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:21.759189+0000 mon.a [INF] from='client.25012 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-4", "tierpool": "test-rados-api-smithi102-80667-4-cache"}]: dispatch 2024-09-11T08:03:22.349 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:21.759524+0000 mon.a [INF] from='client.25195 ' entity='client.admin' cmd=[{"prefix": "osd pool cr api_io_pp: Running main() from gmock_main.cc 2024-09-11T08:03:22.349 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-09-11T08:03:22.349 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [----------] Global test environment set-up. 2024-09-11T08:03:22.349 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-09-11T08:03:22.349 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: seed 80395 2024-09-11T08:03:22.349 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-09-11T08:03:22.349 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-09-11T08:03:22.349 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-09-11T08:03:22.349 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (248 ms) 2024-09-11T08:03:22.349 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-09-11T08:03:22.349 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (13 ms) 2024-09-11T08:03:22.349 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-09-11T08:03:22.350 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (7 ms) 2024-09-11T08:03:22.350 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-09-11T08:03:22.350 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (4 ms) 2024-09-11T08:03:22.350 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-09-11T08:03:22.350 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (6 ms) 2024-09-11T08:03:22.350 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-09-11T08:03:22.350 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (5 ms) 2024-09-11T08:03:22.350 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-09-11T08:03:22.350 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (4 ms) 2024-09-11T08:03:22.350 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-09-11T08:03:22.350 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (5 ms) 2024-09-11T08:03:22.350 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-09-11T08:03:22.351 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (7 ms) 2024-09-11T08:03:22.351 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-09-11T08:03:22.351 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (3 ms) 2024-09-11T08:03:22.351 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-09-11T08:03:22.351 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (7 ms) 2024-09-11T08:03:22.351 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-09-11T08:03:22.351 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (4 ms) 2024-09-11T08:03:22.351 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-09-11T08:03:22.351 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (4 ms) 2024-09-11T08:03:22.351 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-09-11T08:03:22.351 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (5 ms) 2024-09-11T08:03:22.351 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-09-11T08:03:22.351 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (13 ms) 2024-09-11T08:03:22.351 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-09-11T08:03:22.352 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (4 ms) 2024-09-11T08:03:22.352 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-09-11T08:03:22.352 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (6 ms) 2024-09-11T08:03:22.352 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-09-11T08:03:22.352 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (8 ms) 2024-09-11T08:03:22.352 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-09-11T08:03:22.352 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (12 ms) 2024-09-11T08:03:22.352 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (365 ms total) 2024-09-11T08:03:22.352 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: 2024-09-11T08:03:22.352 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-09-11T08:03:22.352 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-09-11T08:03:22.352 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (1051 ms) 2024-09-11T08:03:22.352 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-09-11T08:03:22.353 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (13 ms) 2024-09-11T08:03:22.353 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-09-11T08:03:22.353 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (4 ms) 2024-09-11T08:03:22.353 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-09-11T08:03:22.353 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (2 ms) 2024-09-11T08:03:22.353 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-09-11T08:03:22.353 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (2 ms) 2024-09-11T08:03:22.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: 22.5 deep-scrub starts 2024-09-11T08:03:22.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: 22.5 deep-scrub ok 2024-09-11T08:03:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: 22.4 deep-scrub starts 2024-09-11T08:03:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: 22.4 deep-scrub ok 2024-09-11T08:03:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1614749820' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi102-80356-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-11T08:03:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.25148 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi102-80338-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.25012 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2259835227' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-4", "tierpool": "test-rados-api-smithi102-80667-4-cache"}]: dispatch 2024-09-11T08:03:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: pgmap v77: 548 pgs: 29 creating+peering, 3 unknown, 516 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 442 op/s 2024-09-11T08:03:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.25195 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi102-80461-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.25192 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: osdmap e72: 8 total, 8 up, 8 in 2024-09-11T08:03:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3051776469' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi102-80461-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:22.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1722176585' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi102-80429-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:22.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4227672529' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:22.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/929970514' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:22.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.25124 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi102-80372-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi102-80372-16"}]': finished 2024-09-11T08:03:22.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi102-80756-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi102-80756-7"}]': finished 2024-09-11T08:03:22.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: 22.5 deep-scrub starts 2024-09-11T08:03:22.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: 22.5 deep-scrub ok 2024-09-11T08:03:22.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: 22.4 deep-scrub starts 2024-09-11T08:03:22.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: 22.4 deep-scrub ok 2024-09-11T08:03:22.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1614749820' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi102-80356-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-11T08:03:22.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.25148 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi102-80338-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:22.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.25012 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:22.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2259835227' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-4", "tierpool": "test-rados-api-smithi102-80667-4-cache"}]: dispatch 2024-09-11T08:03:22.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: pgmap v77: 548 pgs: 29 creating+peering, 3 unknown, 516 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 442 op/s 2024-09-11T08:03:22.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:22.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.25195 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi102-80461-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:22.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.25192 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:22.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: osdmap e72: 8 total, 8 up, 8 in 2024-09-11T08:03:22.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3051776469' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi102-80461-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:22.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1722176585' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi102-80429-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:22.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4227672529' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:22.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/929970514' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:22.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.25124 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi102-80372-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi102-80372-16"}]': finished 2024-09-11T08:03:22.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi102-80756-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi102-80756-7"}]': finished 2024-09-11T08:03:22.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.25159 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi102-80885-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi102-80885-6"}]': finished 2024-09-11T08:03:22.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi102-80395-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi102-80395-22"}]': finished 2024-09-11T08:03:22.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: osdmap e73: 8 total, 8 up, 8 in 2024-09-11T08:03:22.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:22.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.25166 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:22.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:22.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.25012 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-4", "tierpool": "test-rados-api-smithi102-80667-4-cache"}]: dispatch 2024-09-11T08:03:22.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.25195 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi102-80461-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:22.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.25192 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi102-80429-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:22.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.25198 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:22.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:22.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1738035696' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-11T08:03:22.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: threexx 2024-09-11T08:03:22.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: from='client.25153 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-11T08:03:22.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[41371]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:22.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.25159 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi102-80885-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi102-80885-6"}]': finished 2024-09-11T08:03:22.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi102-80395-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi102-80395-22"}]': finished 2024-09-11T08:03:22.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: osdmap e73: 8 total, 8 up, 8 in 2024-09-11T08:03:22.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:22.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.25166 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:22.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:22.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.25012 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-4", "tierpool": "test-rados-api-smithi102-80667-4-cache"}]: dispatch 2024-09-11T08:03:22.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.25195 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi102-80461-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:22.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.25192 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi102-80429-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:22.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.25198 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:22.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:22.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1738035696' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-11T08:03:22.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: threexx 2024-09-11T08:03:22.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: from='client.25153 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-11T08:03:22.402 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:22 smithi102 ceph-mon[29849]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: 22.5 deep-scrub starts 2024-09-11T08:03:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: 22.5 deep-scrub ok 2024-09-11T08:03:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: 22.4 deep-scrub starts 2024-09-11T08:03:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: 22.4 deep-scrub ok 2024-09-11T08:03:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1614749820' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi102-80356-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-11T08:03:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.25148 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi102-80338-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.25012 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:22.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2259835227' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-4", "tierpool": "test-rados-api-smithi102-80667-4-cache"}]: dispatch 2024-09-11T08:03:22.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: pgmap v77: 548 pgs: 29 creating+peering, 3 unknown, 516 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 442 op/s 2024-09-11T08:03:22.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:22.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.25195 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi102-80461-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:22.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.25192 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:22.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: osdmap e72: 8 total, 8 up, 8 in 2024-09-11T08:03:22.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3051776469' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi102-80461-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:22.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1722176585' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi102-80429-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:22.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4227672529' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:22.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/929970514' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:22.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.25124 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi102-80372-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi102-80372-16"}]': finished 2024-09-11T08:03:22.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi102-80756-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi102-80756-7"}]': finished 2024-09-11T08:03:22.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.25159 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi102-80885-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi102-80885-6"}]': finished 2024-09-11T08:03:22.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi102-80395-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi102-80395-22"}]': finished 2024-09-11T08:03:22.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: osdmap e73: 8 total, 8 up, 8 in 2024-09-11T08:03:22.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:22.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.25166 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:22.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:22.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.25012 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-4", "tierpool": "test-rados-api-smithi102-80667-4-cache"}]: dispatch 2024-09-11T08:03:22.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.25195 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi102-80461-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:22.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.25192 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi102-80429-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:22.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.25198 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:22.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:22.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1738035696' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-11T08:03:22.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: threexx 2024-09-11T08:03:22.437 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: from='client.25153 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-11T08:03:22.437 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:22 smithi146 ceph-mon[33158]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:23.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[41371]: from='client.25153 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-11T08:03:23.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1738035696' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-11T08:03:23.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[41371]: fourxx 2024-09-11T08:03:23.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[41371]: from='client.25153 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-11T08:03:23.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[41371]: from='client.25166 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:23.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[41371]: from='client.25012 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-4", "tierpool": "test-rados-api-smithi102-80667-4-cache"}]': finished 2024-09-11T08:03:23.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2259835227' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-4", "overlaypool": "test-rados-api-smithi102-80667-4-cache"}]: dispatch 2024-09-11T08:03:23.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2425004462' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:23.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2187970405' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi102-80338-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:23.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-09-11T08:03:23.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:23.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[41371]: pgmap v80: 644 pgs: 29 creating+peering, 131 unknown, 484 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 522 op/s 2024-09-11T08:03:23.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:23.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[41371]: from='client.25198 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:23.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[41371]: from='client.24763 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]': finished 2024-09-11T08:03:23.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[41371]: osdmap e74: 8 total, 8 up, 8 in 2024-09-11T08:03:23.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/929970514' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:23.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[41371]: from='client.25172 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:23.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[41371]: from='client.25178 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi102-80338-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:23.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[41371]: from='client.25012 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-4", "overlaypool": "test-rados-api-smithi102-80667-4-cache"}]: dispatch 2024-09-11T08:03:23.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[41371]: from='client.25166 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:23.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[41371]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:23.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[29849]: from='client.25153 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-11T08:03:23.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1738035696' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-11T08:03:23.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[29849]: fourxx 2024-09-11T08:03:23.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[29849]: from='client.25153 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-11T08:03:23.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[29849]: from='client.25166 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:23.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[29849]: from='client.25012 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-4", "tierpool": "test-rados-api-smithi102-80667-4-cache"}]': finished 2024-09-11T08:03:23.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2259835227' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-4", "overlaypool": "test-rados-api-smithi102-80667-4-cache"}]: dispatch 2024-09-11T08:03:23.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2425004462' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:23.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2187970405' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi102-80338-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:23.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-09-11T08:03:23.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:23.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[29849]: pgmap v80: 644 pgs: 29 creating+peering, 131 unknown, 484 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 522 op/s 2024-09-11T08:03:23.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:23.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[29849]: from='client.25198 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:23.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[29849]: from='client.24763 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]': finished 2024-09-11T08:03:23.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[29849]: osdmap e74: 8 total, 8 up, 8 in 2024-09-11T08:03:23.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/929970514' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:23.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[29849]: from='client.25172 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:23.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[29849]: from='client.25178 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi102-80338-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:23.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[29849]: from='client.25012 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-4", "overlaypool": "test-rados-api-smithi102-80667-4-cache"}]: dispatch 2024-09-11T08:03:23.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[29849]: from='client.25166 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:23.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:23 smithi102 ceph-mon[29849]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:23.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:23 smithi146 ceph-mon[33158]: from='client.25153 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-11T08:03:23.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1738035696' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-11T08:03:23.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:23 smithi146 ceph-mon[33158]: fourxx 2024-09-11T08:03:23.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:23 smithi146 ceph-mon[33158]: from='client.25153 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-11T08:03:23.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:23 smithi146 ceph-mon[33158]: from='client.25166 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:23.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:23 smithi146 ceph-mon[33158]: from='client.25012 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-4", "tierpool": "test-rados-api-smithi102-80667-4-cache"}]': finished 2024-09-11T08:03:23.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2259835227' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-4", "overlaypool": "test-rados-api-smithi102-80667-4-cache"}]: dispatch 2024-09-11T08:03:23.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2425004462' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:23.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2187970405' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi102-80338-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:23.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-09-11T08:03:23.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:23.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:23 smithi146 ceph-mon[33158]: pgmap v80: 644 pgs: 29 creating+peering, 131 unknown, 484 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 522 op/s 2024-09-11T08:03:23.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:23.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:23 smithi146 ceph-mon[33158]: from='client.25198 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:23.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:23 smithi146 ceph-mon[33158]: from='client.24763 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]': finished 2024-09-11T08:03:23.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:23 smithi146 ceph-mon[33158]: osdmap e74: 8 total, 8 up, 8 in 2024-09-11T08:03:23.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/929970514' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:23.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:23 smithi146 ceph-mon[33158]: from='client.25172 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:23.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:23 smithi146 ceph-mon[33158]: from='client.25178 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi102-80338-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:23.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:23 smithi146 ceph-mon[33158]: from='client.25012 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-4", "overlaypool": "test-rados-api-smithi102-80667-4-cache"}]: dispatch 2024-09-11T08:03:23.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:23 smithi146 ceph-mon[33158]: from='client.25166 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:23.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:23 smithi146 ceph-mon[33158]: from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:24.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[29849]: from='client.25153 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-11T08:03:24.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[29849]: from='client.25195 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi102-80461-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi102-80461-10"}]': finished 2024-09-11T08:03:24.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[29849]: from='client.25192 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi102-80429-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi102-80429-10"}]': finished 2024-09-11T08:03:24.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[29849]: from='client.25172 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:24.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[29849]: from='client.25178 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi102-80338-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:24.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[29849]: from='client.25012 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-4", "overlaypool": "test-rados-api-smithi102-80667-4-cache"}]': finished 2024-09-11T08:03:24.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[29849]: from='client.25166 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:24.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[29849]: from='client.24763 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi102-80785-1"}]': finished 2024-09-11T08:03:24.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[29849]: osdmap e75: 8 total, 8 up, 8 in 2024-09-11T08:03:24.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2259835227' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-4-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:03:24.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3089827920' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:24.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[29849]: from='client.25012 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-4-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:03:24.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2807324871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:24.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/637746560' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:24.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3846439703' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:24.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "app2"}]: dispatch 2024-09-11T08:03:24.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:24.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[29849]: from='client.25124 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:24.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[29849]: from='client.25166 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:24.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[29849]: from='client.25159 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:24.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[29849]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:24.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[29849]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:24.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:24.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[41371]: from='client.25153 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-11T08:03:24.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[41371]: from='client.25195 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi102-80461-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi102-80461-10"}]': finished 2024-09-11T08:03:24.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[41371]: from='client.25192 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi102-80429-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi102-80429-10"}]': finished 2024-09-11T08:03:24.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[41371]: from='client.25172 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:24.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[41371]: from='client.25178 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi102-80338-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:24.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[41371]: from='client.25012 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-4", "overlaypool": "test-rados-api-smithi102-80667-4-cache"}]': finished 2024-09-11T08:03:24.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[41371]: from='client.25166 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:24.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[41371]: from='client.24763 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi102-80785-1"}]': finished 2024-09-11T08:03:24.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[41371]: osdmap e75: 8 total, 8 up, 8 in 2024-09-11T08:03:24.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2259835227' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-4-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:03:24.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3089827920' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:24.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[41371]: from='client.25012 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-4-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:03:24.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2807324871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:24.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/637746560' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:24.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3846439703' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:24.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "app2"}]: dispatch 2024-09-11T08:03:24.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:24.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[41371]: from='client.25124 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:24.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[41371]: from='client.25166 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:24.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[41371]: from='client.25159 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:24.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[41371]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:24.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[41371]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:24.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-mon[33158]: from='client.25153 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-11T08:03:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-mon[33158]: from='client.25195 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi102-80461-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi102-80461-10"}]': finished 2024-09-11T08:03:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-mon[33158]: from='client.25192 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi102-80429-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi102-80429-10"}]': finished 2024-09-11T08:03:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-mon[33158]: from='client.25172 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-mon[33158]: from='client.25178 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi102-80338-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-mon[33158]: from='client.25012 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-4", "overlaypool": "test-rados-api-smithi102-80667-4-cache"}]': finished 2024-09-11T08:03:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-mon[33158]: from='client.25166 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-mon[33158]: from='client.24763 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi102-80785-1"}]': finished 2024-09-11T08:03:24.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-mon[33158]: osdmap e75: 8 total, 8 up, 8 in 2024-09-11T08:03:24.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2259835227' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-4-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:03:24.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3089827920' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:24.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-mon[33158]: from='client.25012 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-4-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:03:24.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2807324871' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:24.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/637746560' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:24.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3846439703' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:24.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "app2"}]: dispatch 2024-09-11T08:03:24.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:24.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-mon[33158]: from='client.25124 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:24.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-mon[33158]: from='client.25166 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:24.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-mon[33158]: from='client.25159 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:24.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-mon[33158]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:24.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-mon[33158]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:24.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:24.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:03:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:03:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:03:25.022 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN eate", "pool": "LibRadosLockEC_smithi102-80461-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:25.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:21.759869+0000 mon.a [INF] from='client.25192 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi102-80429-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:25.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:21.760123+0000 mon.a [INF] from='client.25198 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:25.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:21.760262+0000 mon.a [INF] from='client.24763 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi102-80785-1"}]: dispatch 2024-09-11T08:03:25.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:21.786484+0000 mon.b [INF] from='client.? v1:172.21.15.102:0/1738035696' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-11T08:03:25.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:21.786575+0000 client.admin [INF] threexx 2024-09-11T08:03:25.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:21.786679+0000 mon.a [INF] from='client.25153 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-11T08:03:25.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: got: 2024-09-11T08:03:21.796432+0000 mon.a [WRN] Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:25.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (8051 ms) 2024-09-11T08:03:25.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (16704 ms total) 2024-09-11T08:03:25.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: 2024-09-11T08:03:25.024 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: [----------] Global test environment tear-down 2024-09-11T08:03:25.024 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (16704 ms total) 2024-09-11T08:03:25.024 INFO:tasks.workunit.client.0.smithi102.stdout: api_cmd: [ PASSED ] 4 tests. 2024-09-11T08:03:25.313 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-09-11T08:03:25.313 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-09-11T08:03:25.313 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-09-11T08:03:25.313 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-09-11T08:03:25.314 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: seed 80756 2024-09-11T08:03:25.314 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-09-11T08:03:25.314 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (243 ms) 2024-09-11T08:03:25.314 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-09-11T08:03:25.314 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (28 ms) 2024-09-11T08:03:25.314 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-09-11T08:03:25.314 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (84 ms) 2024-09-11T08:03:25.314 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-09-11T08:03:25.314 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (4 ms) 2024-09-11T08:03:25.314 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-09-11T08:03:25.314 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (16 ms) 2024-09-11T08:03:25.314 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (375 ms total) 2024-09-11T08:03:25.314 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: 2024-09-11T08:03:25.315 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-09-11T08:03:25.315 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-09-11T08:03:25.315 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (1044 ms) 2024-09-11T08:03:25.315 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-09-11T08:03:25.315 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (0 ms) 2024-09-11T08:03:25.315 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-09-11T08:03:25.315 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (5 ms) 2024-09-11T08:03:25.315 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-09-11T08:03:25.315 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (8 ms) 2024-09-11T08:03:25.315 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (1057 ms total) 2024-09-11T08:03:25.315 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: 2024-09-11T08:03:25.315 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-09-11T08:03:25.315 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (17046 ms total) 2024-09-11T08:03:25.315 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-09-11T08:03:25.315 INFO:tasks.workunit.client.0.smithi102.stdout:] LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-09-11T08:03:25.316 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (3 ms) 2024-09-11T08:03:25.316 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-09-11T08:03:25.316 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (6 ms) 2024-09-11T08:03:25.316 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-09-11T08:03:25.316 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (4 ms) 2024-09-11T08:03:25.316 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-09-11T08:03:25.316 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (3 ms) 2024-09-11T08:03:25.316 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-09-11T08:03:25.316 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (4 ms) 2024-09-11T08:03:25.316 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-09-11T08:03:25.316 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (3 ms) 2024-09-11T08:03:25.316 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-09-11T08:03:25.316 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (2 ms) 2024-09-11T08:03:25.316 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-09-11T08:03:25.317 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (7 ms) 2024-09-11T08:03:25.317 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-09-11T08:03:25.317 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (4 ms) 2024-09-11T08:03:25.317 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-09-11T08:03:25.317 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (8 ms) 2024-09-11T08:03:25.317 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-09-11T08:03:25.317 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (4 ms) 2024-09-11T08:03:25.317 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-09-11T08:03:25.317 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (5 ms) 2024-09-11T08:03:25.317 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (1125 ms total) 2024-09-11T08:03:25.317 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: 2024-09-11T08:03:25.317 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [----------] Global test environment tear-down 2024-09-11T08:03:25.317 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (17165 ms total) 2024-09-11T08:03:25.317 INFO:tasks.workunit.client.0.smithi102.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-09-11T08:03:25.318 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: Running main() from gmock_main.cc 2024-09-11T08:03:25.318 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [==========] Running 8 tests from 2 test suites. 2024-09-11T08:03:25.318 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [----------] Global test environment set-up. 2024-09-11T08:03:25.318 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [----------] 4 tests from LibRadosStat 2024-09-11T08:03:25.318 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-09-11T08:03:25.318 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [ OK ] LibRadosStat.Stat (242 ms) 2024-09-11T08:03:25.318 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-09-11T08:03:25.318 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [ OK ] LibRadosStat.StatNS (30 ms) 2024-09-11T08:03:25.318 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-09-11T08:03:25.318 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (94 ms) 2024-09-11T08:03:25.318 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-09-11T08:03:25.318 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (4 ms) 2024-09-11T08:03:25.318 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [----------] 4 tests from LibRadosStat (370 ms total) 2024-09-11T08:03:25.318 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: 2024-09-11T08:03:25.319 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-09-11T08:03:25.319 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-09-11T08:03:25.319 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1046 ms) 2024-09-11T08:03:25.319 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-09-11T08:03:25.319 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (10 ms) 2024-09-11T08:03:25.319 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-09-11T08:03:25.319 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (0 ms) 2024-09-11T08:03:25.319 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-09-11T08:03:25.319 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (2 ms) 2024-09-11T08:03:25.319 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (1058 ms total) 2024-09-11T08:03:25.319 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: 2024-09-11T08:03:25.319 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [----------] Global test environment tear-down 2024-09-11T08:03:25.319 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [==========] 8 tests from 2 test suites ran. (17007 ms total) 2024-09-11T08:03:25.319 INFO:tasks.workunit.client.0.smithi102.stdout: api_stat: [ PASSED ] 8 tests. 2024-09-11T08:03:25.320 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: Running main() from gmock_main.cc 2024-09-11T08:03:25.320 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-09-11T08:03:25.320 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [----------] Global test environment set-up. 2024-09-11T08:03:25.320 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-09-11T08:03:25.320 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-09-11T08:03:25.320 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (166 ms) 2024-09-11T08:03:25.320 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-09-11T08:03:25.320 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-09-11T08:03:25.320 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-09-11T08:03:25.320 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: no timeout :/ 2024-09-11T08:03:25.321 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: no timeout :/ 2024-09-11T08:03:25.321 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: no timeout :/ 2024-09-11T08:03:25.321 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: no timeout :/ 2024-09-11T08:03:25.321 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: no timeout :/ 2024-09-11T08:03:25.321 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (202 ms) 2024-09-11T08:03:25.321 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-09-11T08:03:25.321 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (7 ms) 2024-09-11T08:03:25.321 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-09-11T08:03:25.321 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIo.Checksum (3 ms) 2024-09-11T08:03:25.321 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-09-11T08:03:25.321 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (5 ms) 2024-09-11T08:03:25.321 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-09-11T08:03:25.322 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (4 ms) 2024-09-11T08:03:25.322 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-09-11T08:03:25.322 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (5 ms) 2024-09-11T08:03:25.322 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-09-11T08:03:25.322 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (2 ms) 2024-09-11T08:03:25.322 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-09-11T08:03:25.322 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIo.TruncTest (5 ms) 2024-09-11T08:03:25.322 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-09-11T08:03:25.322 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (2 ms) 2024-09-11T08:03:25.322 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-09-11T08:03:25.322 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (4 ms) 2024-09-11T08:03:25.322 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-09-11T08:03:25.323 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIo.RmXattr (9 ms) 2024-09-11T08:03:25.323 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-09-11T08:03:25.323 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIo.XattrIter (4 ms) 2024-09-11T08:03:25.323 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [----------] 14 tests from LibRadosIo (418 ms total) 2024-09-11T08:03:25.323 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: 2024-09-11T08:03:25.323 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-09-11T08:03:25.323 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-09-11T08:03:25.323 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (1041 ms) 2024-09-11T08:03:25.323 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-09-11T08:03:25.323 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (7 ms) 2024-09-11T08:03:25.323 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-09-11T08:03:25.323 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (4 ms) 2024-09-11T08:03:25.324 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-09-11T08:03:25.324 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (5 ms) 2024-09-11T08:03:25.324 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-09-11T08:03:25.324 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (6 ms) 2024-09-11T08:03:25.324 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-09-11T08:03:25.324 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (3 ms) 2024-09-11T08:03:25.324 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-09-11T08:03:25.324 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (3 ms) 2024-09-11T08:03:25.324 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-09-11T08:03:25.324 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (3 ms) 2024-09-11T08:03:25.324 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-09-11T08:03:25.324 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (9 ms) 2024-09-11T08:03:25.324 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-09-11T08:03:25.324 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (4 ms) 2024-09-11T08:03:25.325 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [----------] 10 tests from LibRadosIoEC (1085 ms total) 2024-09-11T08:03:25.325 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: 2024-09-11T08:03:25.325 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [----------] Global test environment tear-down 2024-09-11T08:03:25.325 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [==========] 24 tests from 2 test suites ran. (17179 ms total) 2024-09-11T08:03:25.325 INFO:tasks.workunit.client.0.smithi102.stdout: api_io: [ PASSED ] 24 tests. 2024-09-11T08:03:25.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:03:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.25012 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-4-cache", "mode": "writeback"}]': finished 2024-09-11T08:03:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.25124 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi102-80372-16"}]': finished 2024-09-11T08:03:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.25166 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.25159 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi102-80885-6"}]': finished 2024-09-11T08:03:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7"}]': finished 2024-09-11T08:03:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22"}]': finished 2024-09-11T08:03:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: osdmap e76: 8 total, 8 up, 8 in 2024-09-11T08:03:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3089827920' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2807324871' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/637746560' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3846439703' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi102-80526-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-09-11T08:03:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:03:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.25124 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.25159 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.25166 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:03:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2301110272' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.25193 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1757028854' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:25.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.25231 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:25.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1757028854' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:25.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2259835227' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-4"}]: dispatch 2024-09-11T08:03:25.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: pgmap v83: 564 pgs: 112 unknown, 452 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:03:25.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:25.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.25231 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:25.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.25012 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-4"}]: dispatch 2024-09-11T08:03:25.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1757028854' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:25.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[29849]: from='client.25231 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:25.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:03:25.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.25012 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-4-cache", "mode": "writeback"}]': finished 2024-09-11T08:03:25.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.25124 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi102-80372-16"}]': finished 2024-09-11T08:03:25.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.25166 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:25.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.25159 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi102-80885-6"}]': finished 2024-09-11T08:03:25.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7"}]': finished 2024-09-11T08:03:25.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22"}]': finished 2024-09-11T08:03:25.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: osdmap e76: 8 total, 8 up, 8 in 2024-09-11T08:03:25.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3089827920' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:25.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2807324871' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:25.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/637746560' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:25.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3846439703' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:25.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi102-80526-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-09-11T08:03:25.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:03:25.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:25.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.25124 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:25.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.25159 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:25.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.25166 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:03:25.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:25.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:25.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:25.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2301110272' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:25.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.25193 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:25.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1757028854' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:25.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.25231 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:25.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1757028854' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:25.401 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2259835227' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-4"}]: dispatch 2024-09-11T08:03:25.401 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: pgmap v83: 564 pgs: 112 unknown, 452 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:03:25.401 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:25.401 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.25231 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:25.401 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.25012 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-4"}]: dispatch 2024-09-11T08:03:25.401 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1757028854' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:25.401 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:25 smithi102 ceph-mon[41371]: from='client.25231 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:03:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.25012 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-4-cache", "mode": "writeback"}]': finished 2024-09-11T08:03:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.25124 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi102-80372-16"}]': finished 2024-09-11T08:03:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.25166 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi102-80526-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.25159 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi102-80885-6"}]': finished 2024-09-11T08:03:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi102-80756-7"}]': finished 2024-09-11T08:03:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi102-80395-22"}]': finished 2024-09-11T08:03:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: osdmap e76: 8 total, 8 up, 8 in 2024-09-11T08:03:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3089827920' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2807324871' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/637746560' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3846439703' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi102-80526-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-09-11T08:03:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:03:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.25124 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi102-80372-16"}]: dispatch 2024-09-11T08:03:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.25159 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi102-80885-6"}]: dispatch 2024-09-11T08:03:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.25166 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:03:25.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi102-80756-7"}]: dispatch 2024-09-11T08:03:25.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:25.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi102-80395-22"}]: dispatch 2024-09-11T08:03:25.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2301110272' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:25.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.25193 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:25.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1757028854' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:25.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.25231 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:25.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1757028854' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:25.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2259835227' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-4"}]: dispatch 2024-09-11T08:03:25.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: pgmap v83: 564 pgs: 112 unknown, 452 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:03:25.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:25.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.25231 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:25.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.25012 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-4"}]: dispatch 2024-09-11T08:03:25.437 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1757028854' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:25.437 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:25 smithi146 ceph-mon[33158]: from='client.25231 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:26.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: 22.6 deep-scrub starts 2024-09-11T08:03:26.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: 22.6 deep-scrub ok 2024-09-11T08:03:26.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: from='client.25124 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi102-80372-16"}]': finished 2024-09-11T08:03:26.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: from='client.25159 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi102-80885-6"}]': finished 2024-09-11T08:03:26.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: from='client.25166 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-11T08:03:26.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi102-80756-7"}]': finished 2024-09-11T08:03:26.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:26.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi102-80395-22"}]': finished 2024-09-11T08:03:26.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: from='client.25193 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:26.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: from='client.25012 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-4"}]': finished 2024-09-11T08:03:26.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: from='client.25231 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:26.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: osdmap e77: 8 total, 8 up, 8 in 2024-09-11T08:03:26.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2259835227' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-4", "tierpool": "test-rados-api-smithi102-80667-4-cache"}]: dispatch 2024-09-11T08:03:26.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: from='client.25012 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-4", "tierpool": "test-rados-api-smithi102-80667-4-cache"}]: dispatch 2024-09-11T08:03:26.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1757028854' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi102-80768-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:26.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: from='client.25231 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi102-80768-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:26.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1196407590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:26.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-11T08:03:26.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1620347566' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi102-80338-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:26.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: 22.6 deep-scrub starts 2024-09-11T08:03:26.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: 22.6 deep-scrub ok 2024-09-11T08:03:26.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: from='client.25124 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi102-80372-16"}]': finished 2024-09-11T08:03:26.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: from='client.25159 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi102-80885-6"}]': finished 2024-09-11T08:03:26.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: from='client.25166 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-11T08:03:26.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi102-80756-7"}]': finished 2024-09-11T08:03:26.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:26.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi102-80395-22"}]': finished 2024-09-11T08:03:26.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: from='client.25193 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:26.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: from='client.25012 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-4"}]': finished 2024-09-11T08:03:26.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: from='client.25231 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:26.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: osdmap e77: 8 total, 8 up, 8 in 2024-09-11T08:03:26.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2259835227' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-4", "tierpool": "test-rados-api-smithi102-80667-4-cache"}]: dispatch 2024-09-11T08:03:26.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: from='client.25012 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-4", "tierpool": "test-rados-api-smithi102-80667-4-cache"}]: dispatch 2024-09-11T08:03:26.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1757028854' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi102-80768-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:26.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: from='client.25231 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi102-80768-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:26.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1196407590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:26.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-11T08:03:26.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1620347566' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi102-80338-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:26.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:26.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: from='client.25166 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-11T08:03:26.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "pg_num": 4}]: dispatch 2024-09-11T08:03:26.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "pg_num": 4}]: dispatch 2024-09-11T08:03:26.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:26.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: 22.3 deep-scrub starts 2024-09-11T08:03:26.400 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[41371]: 22.3 deep-scrub ok 2024-09-11T08:03:26.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:26.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: from='client.25166 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-11T08:03:26.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "pg_num": 4}]: dispatch 2024-09-11T08:03:26.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "pg_num": 4}]: dispatch 2024-09-11T08:03:26.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:26.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: 22.3 deep-scrub starts 2024-09-11T08:03:26.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:26 smithi102 ceph-mon[29849]: 22.3 deep-scrub ok 2024-09-11T08:03:26.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: 22.6 deep-scrub starts 2024-09-11T08:03:26.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: 22.6 deep-scrub ok 2024-09-11T08:03:26.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: from='client.25124 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi102-80372-16"}]': finished 2024-09-11T08:03:26.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: from='client.25159 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi102-80885-6"}]': finished 2024-09-11T08:03:26.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: from='client.25166 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-11T08:03:26.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi102-80756-7"}]': finished 2024-09-11T08:03:26.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:26.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi102-80395-22"}]': finished 2024-09-11T08:03:26.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: from='client.25193 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:26.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: from='client.25012 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-4"}]': finished 2024-09-11T08:03:26.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: from='client.25231 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:26.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: osdmap e77: 8 total, 8 up, 8 in 2024-09-11T08:03:26.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2259835227' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-4", "tierpool": "test-rados-api-smithi102-80667-4-cache"}]: dispatch 2024-09-11T08:03:26.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: from='client.25012 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-4", "tierpool": "test-rados-api-smithi102-80667-4-cache"}]: dispatch 2024-09-11T08:03:26.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1757028854' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi102-80768-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:26.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: from='client.25231 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi102-80768-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:26.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1196407590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:26.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-11T08:03:26.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1620347566' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi102-80338-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:26.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:26.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: from='client.25166 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-11T08:03:26.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "pg_num": 4}]: dispatch 2024-09-11T08:03:26.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "pg_num": 4}]: dispatch 2024-09-11T08:03:26.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:26.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: 22.3 deep-scrub starts 2024-09-11T08:03:26.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:26 smithi146 ceph-mon[33158]: 22.3 deep-scrub ok 2024-09-11T08:03:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:03:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[29849]: from='client.25012 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-4", "tierpool": "test-rados-api-smithi102-80667-4-cache"}]': finished 2024-09-11T08:03:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1620347566' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi102-80338-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[29849]: from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[29849]: from='client.25166 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-11T08:03:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[29849]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "pg_num": 4}]': finished 2024-09-11T08:03:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[29849]: osdmap e78: 8 total, 8 up, 8 in 2024-09-11T08:03:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key1"}]: dispatch 2024-09-11T08:03:27.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:27.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[29849]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:27.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[29849]: pool 'PoolQuotaPP_smithi102-80356-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-11T08:03:27.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[29849]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:03:27.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[29849]: from='client.25231 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi102-80768-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]': finished 2024-09-11T08:03:27.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[29849]: osdmap e79: 8 total, 8 up, 8 in 2024-09-11T08:03:27.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[29849]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:27.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[29849]: from='client.25166 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key1"}]: dispatch 2024-09-11T08:03:27.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:03:27.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[41371]: from='client.25012 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-4", "tierpool": "test-rados-api-smithi102-80667-4-cache"}]': finished 2024-09-11T08:03:27.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1620347566' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi102-80338-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:27.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[41371]: from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:27.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[41371]: from='client.25166 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-11T08:03:27.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[41371]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "pg_num": 4}]': finished 2024-09-11T08:03:27.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[41371]: osdmap e78: 8 total, 8 up, 8 in 2024-09-11T08:03:27.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:27.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key1"}]: dispatch 2024-09-11T08:03:27.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:27.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[41371]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:27.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[41371]: pool 'PoolQuotaPP_smithi102-80356-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-11T08:03:27.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[41371]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:03:27.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[41371]: from='client.25231 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi102-80768-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]': finished 2024-09-11T08:03:27.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[41371]: osdmap e79: 8 total, 8 up, 8 in 2024-09-11T08:03:27.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[41371]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:27.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:27 smithi102 ceph-mon[41371]: from='client.25166 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key1"}]: dispatch 2024-09-11T08:03:27.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:27 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:03:27.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:27 smithi146 ceph-mon[33158]: from='client.25012 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-4", "tierpool": "test-rados-api-smithi102-80667-4-cache"}]': finished 2024-09-11T08:03:27.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1620347566' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi102-80338-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:27.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:27 smithi146 ceph-mon[33158]: from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:27.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:27 smithi146 ceph-mon[33158]: from='client.25166 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-11T08:03:27.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:27 smithi146 ceph-mon[33158]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "pg_num": 4}]': finished 2024-09-11T08:03:27.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:27 smithi146 ceph-mon[33158]: osdmap e78: 8 total, 8 up, 8 in 2024-09-11T08:03:27.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:27.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/352083846' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key1"}]: dispatch 2024-09-11T08:03:27.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:27.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:27 smithi146 ceph-mon[33158]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:27.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:27 smithi146 ceph-mon[33158]: pool 'PoolQuotaPP_smithi102-80356-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-11T08:03:27.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:27 smithi146 ceph-mon[33158]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:03:27.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:27 smithi146 ceph-mon[33158]: from='client.25231 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi102-80768-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]': finished 2024-09-11T08:03:27.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:27 smithi146 ceph-mon[33158]: osdmap e79: 8 total, 8 up, 8 in 2024-09-11T08:03:27.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:27 smithi146 ceph-mon[33158]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:27.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:27 smithi146 ceph-mon[33158]: from='client.25166 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key1"}]: dispatch 2024-09-11T08:03:28.900 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[29849]: pgmap v86: 568 pgs: 64 creating+peering, 14 creating+activating, 1 active+clean+snaptrim_wait, 4 unknown, 485 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-09-11T08:03:28.900 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/282007135' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi102-80709-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:28.900 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[29849]: from='client.25217 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi102-80709-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:28.900 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:28.900 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[29849]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:03:28.900 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[29849]: from='client.25166 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key1"}]': finished 2024-09-11T08:03:28.900 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[29849]: from='client.25217 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi102-80709-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:28.901 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[29849]: osdmap e80: 8 total, 8 up, 8 in 2024-09-11T08:03:28.901 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1722176585' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:28.901 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3051776469' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:28.901 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6"}]: dispatch 2024-09-11T08:03:28.901 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3381491160' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:28.901 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[29849]: from='client.25192 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:28.901 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[29849]: from='client.25195 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:28.901 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[29849]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6"}]: dispatch 2024-09-11T08:03:28.901 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[29849]: from='client.25229 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:28.902 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[41371]: pgmap v86: 568 pgs: 64 creating+peering, 14 creating+activating, 1 active+clean+snaptrim_wait, 4 unknown, 485 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-09-11T08:03:28.902 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/282007135' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi102-80709-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:28.902 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[41371]: from='client.25217 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi102-80709-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:28.902 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:28.902 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[41371]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:03:28.902 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[41371]: from='client.25166 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key1"}]': finished 2024-09-11T08:03:28.902 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[41371]: from='client.25217 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi102-80709-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:28.902 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[41371]: osdmap e80: 8 total, 8 up, 8 in 2024-09-11T08:03:28.902 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1722176585' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:28.903 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3051776469' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:28.903 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6"}]: dispatch 2024-09-11T08:03:28.903 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3381491160' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:28.903 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[41371]: from='client.25192 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:28.903 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[41371]: from='client.25195 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:28.903 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[41371]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6"}]: dispatch 2024-09-11T08:03:28.903 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:28 smithi102 ceph-mon[41371]: from='client.25229 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:28.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:28 smithi146 ceph-mon[33158]: pgmap v86: 568 pgs: 64 creating+peering, 14 creating+activating, 1 active+clean+snaptrim_wait, 4 unknown, 485 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-09-11T08:03:28.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/282007135' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi102-80709-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:28.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:28 smithi146 ceph-mon[33158]: from='client.25217 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi102-80709-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:28.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:28.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:28 smithi146 ceph-mon[33158]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:03:28.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:28 smithi146 ceph-mon[33158]: from='client.25166 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi102-80526-1","app":"app1","key":"key1"}]': finished 2024-09-11T08:03:28.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:28 smithi146 ceph-mon[33158]: from='client.25217 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi102-80709-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:28.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:28 smithi146 ceph-mon[33158]: osdmap e80: 8 total, 8 up, 8 in 2024-09-11T08:03:28.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1722176585' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:28.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3051776469' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:28.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6"}]: dispatch 2024-09-11T08:03:28.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3381491160' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:28.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:28 smithi146 ceph-mon[33158]: from='client.25192 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:28.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:28 smithi146 ceph-mon[33158]: from='client.25195 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:28.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:28 smithi146 ceph-mon[33158]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6"}]: dispatch 2024-09-11T08:03:28.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:28 smithi146 ceph-mon[33158]: from='client.25229 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:29.838 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-09-11T08:03:29.838 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-09-11T08:03:29.838 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-09-11T08:03:29.838 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-09-11T08:03:29.838 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: seed 80429 2024-09-11T08:03:29.839 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-09-11T08:03:29.839 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (241 ms) 2024-09-11T08:03:29.839 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-09-11T08:03:29.839 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (4 ms) 2024-09-11T08:03:29.839 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-09-11T08:03:29.839 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1026 ms) 2024-09-11T08:03:29.839 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-09-11T08:03:29.839 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1020 ms) 2024-09-11T08:03:29.839 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-09-11T08:03:29.839 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (4 ms) 2024-09-11T08:03:29.839 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-09-11T08:03:29.839 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (4 ms) 2024-09-11T08:03:29.839 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-09-11T08:03:29.839 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (24 ms) 2024-09-11T08:03:29.840 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-09-11T08:03:29.840 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (5 ms) 2024-09-11T08:03:29.840 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2328 ms total) 2024-09-11T08:03:29.840 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: 2024-09-11T08:03:29.840 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-09-11T08:03:29.840 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-09-11T08:03:29.840 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (1079 ms) 2024-09-11T08:03:29.840 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-09-11T08:03:29.840 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (3 ms) 2024-09-11T08:03:29.840 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-09-11T08:03:29.840 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1004 ms) 2024-09-11T08:03:29.840 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-09-11T08:03:29.840 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1004 ms) 2024-09-11T08:03:29.840 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-09-11T08:03:29.841 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (4 ms) 2024-09-11T08:03:29.841 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-09-11T08:03:29.841 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (4 ms) 2024-09-11T08:03:29.841 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-09-11T08:03:29.841 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (4 ms) 2024-09-11T08:03:29.841 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-09-11T08:03:29.841 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (3 ms) 2024-09-11T08:03:29.841 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (3105 ms total) 2024-09-11T08:03:29.841 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: 2024-09-11T08:03:29.841 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-09-11T08:03:29.841 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (21668 ms total) 2024-09-11T08:03:29.841 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-09-11T08:03:29.850 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:03:29] "GET /metrics HTTP/1.1" 200 251805 "" "Prometheus/2.43.0" 2024-09-11T08:03:29.852 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[29849]: pgmap v89: 496 pgs: 14 creating+activating, 1 active+clean+snaptrim_wait, 76 unknown, 405 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 5.7 KiB/s wr, 4 op/s 2024-09-11T08:03:29.852 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:29.852 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[29849]: from='client.25192 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10"}]': finished 2024-09-11T08:03:29.852 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[29849]: from='client.25195 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi102-80461-10"}]': finished 2024-09-11T08:03:29.852 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[29849]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6"}]': finished 2024-09-11T08:03:29.852 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[29849]: from='client.25229 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:29.853 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[29849]: osdmap e81: 8 total, 8 up, 8 in 2024-09-11T08:03:29.853 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3051776469' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:29.853 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1722176585' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:29.853 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1757028854' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:29.853 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2382410648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:29.853 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1787162934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi102-80338-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:29.853 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "mode": "writeback"}]: dispatch 2024-09-11T08:03:29.853 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[29849]: from='client.25195 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:29.853 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[29849]: from='client.25192 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:29.853 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[29849]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "mode": "writeback"}]: dispatch 2024-09-11T08:03:29.853 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[29849]: from='client.25231 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:29.854 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[29849]: from='client.25252 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:29.854 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[29849]: from='client.25255 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi102-80338-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:29.854 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:03:29.854 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:03:29.854 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[41371]: pgmap v89: 496 pgs: 14 creating+activating, 1 active+clean+snaptrim_wait, 76 unknown, 405 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 5.7 KiB/s wr, 4 op/s 2024-09-11T08:03:29.854 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:29.854 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[41371]: from='client.25192 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10"}]': finished 2024-09-11T08:03:29.854 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[41371]: from='client.25195 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi102-80461-10"}]': finished 2024-09-11T08:03:29.854 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[41371]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6"}]': finished 2024-09-11T08:03:29.855 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[41371]: from='client.25229 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:29.855 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[41371]: osdmap e81: 8 total, 8 up, 8 in 2024-09-11T08:03:29.855 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3051776469' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:29.855 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1722176585' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:29.855 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1757028854' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:29.855 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2382410648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:29.855 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1787162934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi102-80338-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:29.855 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "mode": "writeback"}]: dispatch 2024-09-11T08:03:29.855 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[41371]: from='client.25195 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:29.855 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[41371]: from='client.25192 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:29.855 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[41371]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "mode": "writeback"}]: dispatch 2024-09-11T08:03:29.855 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[41371]: from='client.25231 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:29.856 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[41371]: from='client.25252 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:29.856 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[41371]: from='client.25255 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi102-80338-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:29.856 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:03:29.856 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:29 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:03:29.856 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: Running main() from gmock_main.cc 2024-09-11T08:03:29.856 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-09-11T08:03:29.856 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [----------] Global test environment set-up. 2024-09-11T08:03:29.856 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-09-11T08:03:29.856 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-09-11T08:03:29.857 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (275 ms) 2024-09-11T08:03:29.857 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-09-11T08:03:29.857 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ OK ] LibRadosLock.LockShared (5 ms) 2024-09-11T08:03:29.857 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-09-11T08:03:29.857 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1005 ms) 2024-09-11T08:03:29.857 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-09-11T08:03:29.857 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1045 ms) 2024-09-11T08:03:29.857 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-09-11T08:03:29.857 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (44 ms) 2024-09-11T08:03:29.857 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-09-11T08:03:29.857 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ OK ] LibRadosLock.Unlock (17 ms) 2024-09-11T08:03:29.857 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-09-11T08:03:29.857 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (58 ms) 2024-09-11T08:03:29.858 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-09-11T08:03:29.858 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (3 ms) 2024-09-11T08:03:29.858 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [----------] 8 tests from LibRadosLock (2452 ms total) 2024-09-11T08:03:29.858 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: 2024-09-11T08:03:29.858 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-09-11T08:03:29.858 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-09-11T08:03:29.858 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (1075 ms) 2024-09-11T08:03:29.858 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-09-11T08:03:29.858 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (4 ms) 2024-09-11T08:03:29.858 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-09-11T08:03:29.858 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1004 ms) 2024-09-11T08:03:29.858 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-09-11T08:03:29.858 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1005 ms) 2024-09-11T08:03:29.859 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-09-11T08:03:29.859 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (3 ms) 2024-09-11T08:03:29.859 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-09-11T08:03:29.859 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (4 ms) 2024-09-11T08:03:29.859 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-09-11T08:03:29.859 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (5 ms) 2024-09-11T08:03:29.859 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-09-11T08:03:29.859 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (3 ms) 2024-09-11T08:03:29.859 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (3103 ms total) 2024-09-11T08:03:29.859 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: 2024-09-11T08:03:29.859 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [----------] Global test environment tear-down 2024-09-11T08:03:29.859 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (21671 ms total) 2024-09-11T08:03:29.859 INFO:tasks.workunit.client.0.smithi102.stdout: api_lock: [ PASSED ] 16 tests. 2024-09-11T08:03:29.933 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:03:29 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:03:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:29 smithi146 ceph-mon[33158]: pgmap v89: 496 pgs: 14 creating+activating, 1 active+clean+snaptrim_wait, 76 unknown, 405 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 5.7 KiB/s wr, 4 op/s 2024-09-11T08:03:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:29 smithi146 ceph-mon[33158]: from='client.25192 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi102-80429-10"}]': finished 2024-09-11T08:03:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:29 smithi146 ceph-mon[33158]: from='client.25195 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi102-80461-10"}]': finished 2024-09-11T08:03:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:29 smithi146 ceph-mon[33158]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6"}]': finished 2024-09-11T08:03:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:29 smithi146 ceph-mon[33158]: from='client.25229 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:29 smithi146 ceph-mon[33158]: osdmap e81: 8 total, 8 up, 8 in 2024-09-11T08:03:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3051776469' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:29.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1722176585' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:29.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1757028854' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:29.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2382410648' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:29.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1787162934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi102-80338-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:29.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "mode": "writeback"}]: dispatch 2024-09-11T08:03:29.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:29 smithi146 ceph-mon[33158]: from='client.25195 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi102-80461-10"}]: dispatch 2024-09-11T08:03:29.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:29 smithi146 ceph-mon[33158]: from='client.25192 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi102-80429-10"}]: dispatch 2024-09-11T08:03:29.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:29 smithi146 ceph-mon[33158]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "mode": "writeback"}]: dispatch 2024-09-11T08:03:29.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:29 smithi146 ceph-mon[33158]: from='client.25231 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:29.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:29 smithi146 ceph-mon[33158]: from='client.25252 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:29.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:29 smithi146 ceph-mon[33158]: from='client.25255 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi102-80338-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:29.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:29 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:03:29.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:29 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:03:30.889 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: Running main() from gmock_main.cc 2024-09-11T08:03:30.889 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-09-11T08:03:30.889 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-09-11T08:03:30.889 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-09-11T08:03:30.889 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-09-11T08:03:30.889 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: watch_notify_test_cb 2024-09-11T08:03:30.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (281 ms) 2024-09-11T08:03:30.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-09-11T08:03:30.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94332992131744 err -107 2024-09-11T08:03:30.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-09-11T08:03:30.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (9 ms) 2024-09-11T08:03:30.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-09-11T08:03:30.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94332992312880 err -107 2024-09-11T08:03:30.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-09-11T08:03:30.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (8 ms) 2024-09-11T08:03:30.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-09-11T08:03:30.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: watch_notify2_test_cb from 24754 notify_id 287762808832 cookie 94332992336272 2024-09-11T08:03:30.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (7 ms) 2024-09-11T08:03:30.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-09-11T08:03:30.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: watch_notify2_test_cb from 24754 notify_id 287762808832 cookie 94332992365312 2024-09-11T08:03:30.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (8 ms) 2024-09-11T08:03:30.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-09-11T08:03:30.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: watch_notify2_test_cb from 24754 notify_id 287762808832 cookie 94332992341824 2024-09-11T08:03:30.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (8 ms) 2024-09-11T08:03:30.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-09-11T08:03:30.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: watch_notify2_test_cb from 24754 notify_id 287762808832 cookie 94332992341824 2024-09-11T08:03:30.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: watch_notify2_test_cb from 24754 notify_id 287762808832 cookie 94332992380320 2024-09-11T08:03:30.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (10 ms) 2024-09-11T08:03:30.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-09-11T08:03:30.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: watch_notify2_test_cb from 24754 notify_id 287762808833 cookie 94332992341824 2024-09-11T08:03:30.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: watch_notify2_test_cb from 24754 notify_id 292057776130 cookie 94332992341824 2024-09-11T08:03:30.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3005 ms) 2024-09-11T08:03:30.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-09-11T08:03:30.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-09-11T08:03:30.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94332992380320 err -107 2024-09-11T08:03:30.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: watch_notify2_test_cb from 24754 notify_id 322122547202 cookie 94332992380320 2024-09-11T08:03:30.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5062 ms) 2024-09-11T08:03:30.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-09-11T08:03:30.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94332992380320 err -107 2024-09-11T08:03:30.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-09-11T08:03:30.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (8 ms) 2024-09-11T08:03:30.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (8407 ms total) 2024-09-11T08:03:30.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: 2024-09-11T08:03:30.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-09-11T08:03:30.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-09-11T08:03:30.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: watch_notify_test_cb 2024-09-11T08:03:30.893 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1088 ms) 2024-09-11T08:03:30.893 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1088 ms total) 2024-09-11T08:03:30.893 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: 2024-09-11T08:03:30.893 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-09-11T08:03:30.893 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (22617 ms total) 2024-09-11T08:03:30.893 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-09-11T08:03:30.904 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:03:30.904 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:30.904 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:03:30.904 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[41371]: from='client.25195 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi102-80461-10"}]': finished 2024-09-11T08:03:30.905 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[41371]: from='client.25192 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi102-80429-10"}]': finished 2024-09-11T08:03:30.905 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[41371]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "mode": "writeback"}]': finished 2024-09-11T08:03:30.905 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[41371]: from='client.25231 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]': finished 2024-09-11T08:03:30.905 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[41371]: from='client.25252 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:30.905 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[41371]: from='client.25255 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi102-80338-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:30.905 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[41371]: osdmap e82: 8 total, 8 up, 8 in 2024-09-11T08:03:30.905 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:30.905 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:30.905 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1757028854' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:30.905 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[41371]: from='client.25231 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:30.906 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:03:30.906 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:30.906 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:03:30.906 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[29849]: from='client.25195 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi102-80461-10"}]': finished 2024-09-11T08:03:30.906 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[29849]: from='client.25192 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi102-80429-10"}]': finished 2024-09-11T08:03:30.906 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[29849]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "mode": "writeback"}]': finished 2024-09-11T08:03:30.906 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[29849]: from='client.25231 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]': finished 2024-09-11T08:03:30.906 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[29849]: from='client.25252 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:30.906 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[29849]: from='client.25255 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi102-80338-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:30.906 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[29849]: osdmap e82: 8 total, 8 up, 8 in 2024-09-11T08:03:30.907 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:30.907 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:30.907 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1757028854' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:30.907 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:30 smithi102 ceph-mon[29849]: from='client.25231 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:30.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:30 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:03:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:30 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:30 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:03:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:30 smithi146 ceph-mon[33158]: from='client.25195 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi102-80461-10"}]': finished 2024-09-11T08:03:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:30 smithi146 ceph-mon[33158]: from='client.25192 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi102-80429-10"}]': finished 2024-09-11T08:03:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:30 smithi146 ceph-mon[33158]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "mode": "writeback"}]': finished 2024-09-11T08:03:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:30 smithi146 ceph-mon[33158]: from='client.25231 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi102-80768-12"}]': finished 2024-09-11T08:03:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:30 smithi146 ceph-mon[33158]: from='client.25252 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:30 smithi146 ceph-mon[33158]: from='client.25255 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi102-80338-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:30 smithi146 ceph-mon[33158]: osdmap e82: 8 total, 8 up, 8 in 2024-09-11T08:03:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:30 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:30 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:30.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:30 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1757028854' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:30.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:30 smithi146 ceph-mon[33158]: from='client.25231 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi102-80768-12"}]: dispatch 2024-09-11T08:03:32.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:31 smithi102 ceph-mon[29849]: pgmap v92: 520 pgs: 32 creating+peering, 488 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.5 KiB/s wr, 4 op/s 2024-09-11T08:03:32.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:31 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:32.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:31 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:32.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:31 smithi102 ceph-mon[29849]: from='client.25231 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi102-80768-12"}]': finished 2024-09-11T08:03:32.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:31 smithi102 ceph-mon[29849]: osdmap e83: 8 total, 8 up, 8 in 2024-09-11T08:03:32.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:31 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2818256222' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:32.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:31 smithi102 ceph-mon[41371]: pgmap v92: 520 pgs: 32 creating+peering, 488 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.5 KiB/s wr, 4 op/s 2024-09-11T08:03:32.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:31 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:32.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:31 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:32.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:31 smithi102 ceph-mon[41371]: from='client.25231 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi102-80768-12"}]': finished 2024-09-11T08:03:32.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:31 smithi102 ceph-mon[41371]: osdmap e83: 8 total, 8 up, 8 in 2024-09-11T08:03:32.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:31 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2818256222' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:32.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:31 smithi146 ceph-mon[33158]: pgmap v92: 520 pgs: 32 creating+peering, 488 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.5 KiB/s wr, 4 op/s 2024-09-11T08:03:32.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:31 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:32.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:31 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:32.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:31 smithi146 ceph-mon[33158]: from='client.25231 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi102-80768-12"}]': finished 2024-09-11T08:03:32.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:31 smithi146 ceph-mon[33158]: osdmap e83: 8 total, 8 up, 8 in 2024-09-11T08:03:32.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:31 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2818256222' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:33.060 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-09-11T08:03:33.061 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-09-11T08:03:33.061 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-09-11T08:03:33.061 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-09-11T08:03:33.061 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-09-11T08:03:33.061 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: notify 2024-09-11T08:03:33.061 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (1140 ms) 2024-09-11T08:03:33.061 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-09-11T08:03:33.061 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (1048 ms) 2024-09-11T08:03:33.061 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (2188 ms total) 2024-09-11T08:03:33.061 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: 2024-09-11T08:03:33.061 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-09-11T08:03:33.062 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-09-11T08:03:33.062 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: notify 2024-09-11T08:03:33.062 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (36 ms) 2024-09-11T08:03:33.062 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-09-11T08:03:33.062 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: notify 2024-09-11T08:03:33.062 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (4546 ms) 2024-09-11T08:03:33.062 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-09-11T08:03:33.062 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (25 ms) 2024-09-11T08:03:33.062 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-09-11T08:03:33.062 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (34 ms) 2024-09-11T08:03:33.062 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-09-11T08:03:33.062 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: handle_notify cookie 94152800619520 notify_id 352187318274 notifier_gid 25187 2024-09-11T08:03:33.062 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (15 ms) 2024-09-11T08:03:33.062 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-09-11T08:03:33.062 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: handle_notify cookie 94152800462336 notify_id 352187318275 notifier_gid 25187 2024-09-11T08:03:33.063 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (12 ms) 2024-09-11T08:03:33.063 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-09-11T08:03:33.063 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: handle_notify cookie 94152800462336 notify_id 352187318275 notifier_gid 25187 2024-09-11T08:03:33.063 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (18 ms) 2024-09-11T08:03:33.063 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-09-11T08:03:33.063 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: handle_notify cookie 94152800462336 notify_id 352187318273 notifier_gid 25187 2024-09-11T08:03:33.063 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (12 ms) 2024-09-11T08:03:33.063 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-09-11T08:03:33.063 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: handle_notify cookie 94152800462336 notify_id 352187318276 notifier_gid 25187 2024-09-11T08:03:33.063 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (7 ms) 2024-09-11T08:03:33.063 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-09-11T08:03:33.063 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: handle_notify cookie 94152800490880 notify_id 352187318276 notifier_gid 25187 2024-09-11T08:03:33.063 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (28 ms) 2024-09-11T08:03:33.064 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-09-11T08:03:33.064 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: trying... 2024-09-11T08:03:33.064 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: handle_notify cookie 94152800490880 notify_id 352187318273 notifier_gid 25187 2024-09-11T08:03:33.064 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: timed out 2024-09-11T08:03:33.064 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: flushing 2024-09-11T08:03:33.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:33.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:32 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:33.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2818256222' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:33.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3863181413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi102-80338-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:33.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/12565684' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi102-80697-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:33.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:33.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:32 smithi102 ceph-mon[29849]: osdmap e84: 8 total, 8 up, 8 in 2024-09-11T08:03:33.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:32 smithi102 ceph-mon[29849]: from='client.28414 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi102-80338-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:33.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:32 smithi102 ceph-mon[29849]: from='client.28396 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi102-80697-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:33.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:32 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:33.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:33.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:33.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:32 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:33.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2818256222' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:33.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3863181413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi102-80338-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:33.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/12565684' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi102-80697-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:33.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:33.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:32 smithi102 ceph-mon[41371]: osdmap e84: 8 total, 8 up, 8 in 2024-09-11T08:03:33.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:32 smithi102 ceph-mon[41371]: from='client.28414 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi102-80338-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:33.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:32 smithi102 ceph-mon[41371]: from='client.28396 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi102-80697-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:33.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:32 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:33.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:32 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2818256222' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3863181413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi102-80338-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/12565684' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi102-80697-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:32 smithi146 ceph-mon[33158]: osdmap e84: 8 total, 8 up, 8 in 2024-09-11T08:03:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:32 smithi146 ceph-mon[33158]: from='client.28414 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi102-80338-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:32 smithi146 ceph-mon[33158]: from='client.28396 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi102-80697-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:32 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:34.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:33 smithi102 ceph-mon[41371]: pgmap v95: 552 pgs: 128 unknown, 32 creating+peering, 392 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.7 KiB/s wr, 4 op/s 2024-09-11T08:03:34.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:33 smithi102 ceph-mon[41371]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:03:34.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:33 smithi102 ceph-mon[41371]: from='client.28414 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi102-80338-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:34.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:33 smithi102 ceph-mon[41371]: from='client.28396 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi102-80697-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:34.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:33 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:34.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:33 smithi102 ceph-mon[41371]: osdmap e85: 8 total, 8 up, 8 in 2024-09-11T08:03:34.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1709110524' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:34.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:33 smithi102 ceph-mon[41371]: from='client.28322 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:34.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:34.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:33 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:34.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:34.151 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:33 smithi102 ceph-mon[29849]: pgmap v95: 552 pgs: 128 unknown, 32 creating+peering, 392 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.7 KiB/s wr, 4 op/s 2024-09-11T08:03:34.151 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:33 smithi102 ceph-mon[29849]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:03:34.151 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:33 smithi102 ceph-mon[29849]: from='client.28414 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi102-80338-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:34.151 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:33 smithi102 ceph-mon[29849]: from='client.28396 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi102-80697-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:34.151 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:33 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:34.151 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:33 smithi102 ceph-mon[29849]: osdmap e85: 8 total, 8 up, 8 in 2024-09-11T08:03:34.151 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1709110524' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:34.151 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:33 smithi102 ceph-mon[29849]: from='client.28322 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:34.151 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:34.151 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:33 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:34.151 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:34.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:33 smithi146 ceph-mon[33158]: pgmap v95: 552 pgs: 128 unknown, 32 creating+peering, 392 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.7 KiB/s wr, 4 op/s 2024-09-11T08:03:34.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:33 smithi146 ceph-mon[33158]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:03:34.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:33 smithi146 ceph-mon[33158]: from='client.28414 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi102-80338-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:34.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:33 smithi146 ceph-mon[33158]: from='client.28396 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi102-80697-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:34.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:33 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:34.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:33 smithi146 ceph-mon[33158]: osdmap e85: 8 total, 8 up, 8 in 2024-09-11T08:03:34.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1709110524' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:34.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:33 smithi146 ceph-mon[33158]: from='client.28322 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:34.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:34.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:33 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:34.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:34.925 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:03:34 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:03:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:03:35.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:34 smithi146 ceph-mon[33158]: from='client.28322 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:35.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:34 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:03:35.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:34 smithi146 ceph-mon[33158]: osdmap e86: 8 total, 8 up, 8 in 2024-09-11T08:03:35.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:34 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-7"}]: dispatch 2024-09-11T08:03:35.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:34 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-7"}]: dispatch 2024-09-11T08:03:35.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:34 smithi146 ceph-mon[33158]: pgmap v98: 488 pgs: 128 unknown, 32 creating+peering, 328 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:03:35.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:34 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:34 smithi102 ceph-mon[29849]: from='client.28322 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:34 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:03:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:34 smithi102 ceph-mon[29849]: osdmap e86: 8 total, 8 up, 8 in 2024-09-11T08:03:35.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:34 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-7"}]: dispatch 2024-09-11T08:03:35.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:34 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-7"}]: dispatch 2024-09-11T08:03:35.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:34 smithi102 ceph-mon[29849]: pgmap v98: 488 pgs: 128 unknown, 32 creating+peering, 328 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:03:35.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:34 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:35.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:34 smithi102 ceph-mon[41371]: from='client.28322 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:35.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:34 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:03:35.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:34 smithi102 ceph-mon[41371]: osdmap e86: 8 total, 8 up, 8 in 2024-09-11T08:03:35.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:34 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-7"}]: dispatch 2024-09-11T08:03:35.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:34 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-7"}]: dispatch 2024-09-11T08:03:35.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:34 smithi102 ceph-mon[41371]: pgmap v98: 488 pgs: 128 unknown, 32 creating+peering, 328 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:03:35.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:34 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:35.625 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: [==========] Running 4 tests from 1 test suite. 2024-09-11T08:03:35.625 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: [----------] Global test environment set-up. 2024-09-11T08:03:35.625 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: [----------] 4 tests from LibRadosService 2024-09-11T08:03:35.625 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-09-11T08:03:35.625 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5039 ms) 2024-09-11T08:03:35.625 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-09-11T08:03:35.625 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: [ OK ] LibRadosService.RegisterLate (18 ms) 2024-09-11T08:03:35.625 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-09-11T08:03:35.625 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: cluster: 2024-09-11T08:03:35.625 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: id: 2051892c-7013-11ef-bcea-c7b262605968 2024-09-11T08:03:35.626 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: health: HEALTH_OK 2024-09-11T08:03:35.626 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: 2024-09-11T08:03:35.626 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: services: 2024-09-11T08:03:35.626 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-09-11T08:03:35.626 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: mgr: y(active, since 119s), standbys: x 2024-09-11T08:03:35.626 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 3m) 2024-09-11T08:03:35.626 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-09-11T08:03:35.626 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: 2024-09-11T08:03:35.626 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: data: 2024-09-11T08:03:35.626 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: pools: 6 pools, 132 pgs 2024-09-11T08:03:35.626 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: objects: 196 objects, 582 KiB 2024-09-11T08:03:35.626 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:03:35.626 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: pgs: 132 active+clean 2024-09-11T08:03:35.626 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: 2024-09-11T08:03:35.626 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: io: 2024-09-11T08:03:35.627 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: client: 853 B/s rd, 0 op/s rd, 0 op/s wr 2024-09-11T08:03:35.627 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: 2024-09-11T08:03:35.627 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: 2024-09-11T08:03:35.627 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: cluster: 2024-09-11T08:03:35.627 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: id: 2051892c-7013-11ef-bcea-c7b262605968 2024-09-11T08:03:35.627 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: health: HEALTH_WARN 2024-09-11T08:03:35.627 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: 22 pool(s) do not have an application enabled 2024-09-11T08:03:35.627 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: 2024-09-11T08:03:35.627 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: services: 2024-09-11T08:03:35.627 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-09-11T08:03:35.627 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-09-11T08:03:35.627 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 3m) 2024-09-11T08:03:35.628 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-09-11T08:03:35.628 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-09-11T08:03:35.628 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-09-11T08:03:35.628 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: 2024-09-11T08:03:35.628 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: data: 2024-09-11T08:03:35.628 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: pools: 30 pools, 900 pgs 2024-09-11T08:03:35.628 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: objects: 196 objects, 582 KiB 2024-09-11T08:03:35.628 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:03:35.628 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: pgs: 79.000% pgs unknown 2024-09-11T08:03:35.628 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: 6.333% pgs not active 2024-09-11T08:03:35.628 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: 711 unknown 2024-09-11T08:03:35.628 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: 132 active+clean 2024-09-11T08:03:35.628 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: 57 creating+peering 2024-09-11T08:03:35.629 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: 2024-09-11T08:03:35.629 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: io: 2024-09-11T08:03:35.629 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: client: 853 B/s rd, 0 op/s rd, 0 op/s wr 2024-09-11T08:03:35.629 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: 2024-09-11T08:03:35.629 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: 2024-09-11T08:03:35.629 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: [ OK ] LibRadosService.StatusFormat (2178 ms) 2024-09-11T08:03:35.629 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: [ RUN ] LibRadosService.Status 2024-09-11T08:03:35.629 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: [ OK ] LibRadosService.Status (20027 ms) 2024-09-11T08:03:35.629 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: [----------] 4 tests from LibRadosService (27262 ms total) 2024-09-11T08:03:35.629 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: 2024-09-11T08:03:35.629 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: [----------] Global test environment tear-down 2024-09-11T08:03:35.629 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: [==========] 4 tests from 1 test suite ran. (27262 ms total) 2024-09-11T08:03:35.629 INFO:tasks.workunit.client.0.smithi102.stdout: api_service: [ PASSED ] 4 tests. 2024-09-11T08:03:36.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-7"}]': finished 2024-09-11T08:03:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[29849]: osdmap e87: 8 total, 8 up, 8 in 2024-09-11T08:03:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/431154845' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi102-80338-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[29849]: from='client.31792 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi102-80338-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3533043841' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi102-80356-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[29849]: from='client.31565 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi102-80356-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:03:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:03:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[29849]: from='client.31792 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi102-80338-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[29849]: from='client.31565 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi102-80356-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:36.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:03:36.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[29849]: osdmap e88: 8 total, 8 up, 8 in 2024-09-11T08:03:36.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4185037972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:36.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-7"}]: dispatch 2024-09-11T08:03:36.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2933593694' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:36.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[29849]: from='client.32992 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:36.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-7"}]: dispatch 2024-09-11T08:03:36.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-7"}]': finished 2024-09-11T08:03:36.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[41371]: osdmap e87: 8 total, 8 up, 8 in 2024-09-11T08:03:36.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/431154845' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi102-80338-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:36.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[41371]: from='client.31792 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi102-80338-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:36.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3533043841' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi102-80356-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:36.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[41371]: from='client.31565 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi102-80356-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:36.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:03:36.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:03:36.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:36.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[41371]: from='client.31792 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi102-80338-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:36.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[41371]: from='client.31565 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi102-80356-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:36.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:03:36.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[41371]: osdmap e88: 8 total, 8 up, 8 in 2024-09-11T08:03:36.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4185037972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:36.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-7"}]: dispatch 2024-09-11T08:03:36.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2933593694' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:36.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[41371]: from='client.32992 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:36.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:35 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-7"}]: dispatch 2024-09-11T08:03:36.432 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:35 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-7"}]': finished 2024-09-11T08:03:36.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:35 smithi146 ceph-mon[33158]: osdmap e87: 8 total, 8 up, 8 in 2024-09-11T08:03:36.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/431154845' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi102-80338-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:36.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:35 smithi146 ceph-mon[33158]: from='client.31792 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi102-80338-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:36.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3533043841' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi102-80356-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:36.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:35 smithi146 ceph-mon[33158]: from='client.31565 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi102-80356-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:36.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:03:36.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:35 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:03:36.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:36.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:35 smithi146 ceph-mon[33158]: from='client.31792 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi102-80338-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:36.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:35 smithi146 ceph-mon[33158]: from='client.31565 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi102-80356-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:36.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:35 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:03:36.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:35 smithi146 ceph-mon[33158]: osdmap e88: 8 total, 8 up, 8 in 2024-09-11T08:03:36.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4185037972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-7"}]: dispatch 2024-09-11T08:03:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2933593694' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:35 smithi146 ceph-mon[33158]: from='client.32992 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:35 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-7"}]: dispatch 2024-09-11T08:03:37.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:37 smithi102 ceph-mon[29849]: from='client.32983 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:37.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:37 smithi102 ceph-mon[29849]: pgmap v101: 552 pgs: 64 unknown, 64 creating+peering, 424 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-11T08:03:37.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:37 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:37.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:37.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:37 smithi102 ceph-mon[29849]: from='client.32992 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:37.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:37 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-7"}]': finished 2024-09-11T08:03:37.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:37 smithi102 ceph-mon[29849]: from='client.32983 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:37.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:37 smithi102 ceph-mon[29849]: osdmap e89: 8 total, 8 up, 8 in 2024-09-11T08:03:37.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:37 smithi102 ceph-mon[41371]: from='client.32983 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:37.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:37 smithi102 ceph-mon[41371]: pgmap v101: 552 pgs: 64 unknown, 64 creating+peering, 424 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-11T08:03:37.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:37 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:37.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:37.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:37 smithi102 ceph-mon[41371]: from='client.32992 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:37.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:37 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-7"}]': finished 2024-09-11T08:03:37.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:37 smithi102 ceph-mon[41371]: from='client.32983 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:37.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:37 smithi102 ceph-mon[41371]: osdmap e89: 8 total, 8 up, 8 in 2024-09-11T08:03:37.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:37 smithi146 ceph-mon[33158]: from='client.32983 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:37.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:37 smithi146 ceph-mon[33158]: pgmap v101: 552 pgs: 64 unknown, 64 creating+peering, 424 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-11T08:03:37.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:37 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:03:37.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:37.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:37 smithi146 ceph-mon[33158]: from='client.32992 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:37.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:37 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-7"}]': finished 2024-09-11T08:03:37.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:37 smithi146 ceph-mon[33158]: from='client.32983 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-81036-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:37.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:37 smithi146 ceph-mon[33158]: osdmap e89: 8 total, 8 up, 8 in 2024-09-11T08:03:38.616 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-09-11T08:03:38.617 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: [----------] Global test environment set-up. 2024-09-11T08:03:38.617 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-09-11T08:03:38.617 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-09-11T08:03:38.617 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5050 ms) 2024-09-11T08:03:38.617 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-09-11T08:03:38.617 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (40 ms) 2024-09-11T08:03:38.617 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-09-11T08:03:38.617 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20035 ms) 2024-09-11T08:03:38.617 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-09-11T08:03:38.617 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: attempt 0 of 20 2024-09-11T08:03:38.617 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: failed to deregister: 2024-09-11T08:03:38.617 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: { 2024-09-11T08:03:38.618 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "epoch": 19, 2024-09-11T08:03:38.618 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "modified": "2024-09-11T08:03:36.366211+0000", 2024-09-11T08:03:38.618 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "services": { 2024-09-11T08:03:38.618 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "laundry": { 2024-09-11T08:03:38.618 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "daemons": { 2024-09-11T08:03:38.618 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "summary": "", 2024-09-11T08:03:38.618 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "close-test-pid81007": { 2024-09-11T08:03:38.618 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "start_epoch": 18, 2024-09-11T08:03:38.618 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "start_stamp": "2024-09-11T08:03:33.581323+0000", 2024-09-11T08:03:38.618 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "gid": 31154, 2024-09-11T08:03:38.618 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "addr": "172.21.15.102:0/2859043523", 2024-09-11T08:03:38.618 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "metadata": { 2024-09-11T08:03:38.619 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "arch": "x86_64", 2024-09-11T08:03:38.619 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "ceph_release": "quincy", 2024-09-11T08:03:38.619 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T08:03:38.619 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "ceph_version_short": "17.2.7-1623-gd0e6828a", 2024-09-11T08:03:38.619 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-09-11T08:03:38.619 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "distro": "centos", 2024-09-11T08:03:38.619 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-09-11T08:03:38.619 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "distro_version": "9", 2024-09-11T08:03:38.619 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "foo": "bar", 2024-09-11T08:03:38.619 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "hostname": "smithi102", 2024-09-11T08:03:38.619 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024", 2024-09-11T08:03:38.619 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "kernel_version": "5.14.0-505.el9.x86_64", 2024-09-11T08:03:38.619 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "mem_swap_kb": "0", 2024-09-11T08:03:38.620 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "mem_total_kb": "32487292", 2024-09-11T08:03:38.620 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "os": "Linux", 2024-09-11T08:03:38.620 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "this": "that" 2024-09-11T08:03:38.620 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: }, 2024-09-11T08:03:38.620 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "task_status": {} 2024-09-11T08:03:38.620 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: } 2024-09-11T08:03:38.620 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: } 2024-09-11T08:03:38.620 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: }, 2024-09-11T08:03:38.620 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "rgw": { 2024-09-11T08:03:38.620 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "daemons": { 2024-09-11T08:03:38.620 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "summary": "", 2024-09-11T08:03:38.620 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "24395": { 2024-09-11T08:03:38.620 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "start_epoch": 12, 2024-09-11T08:03:38.621 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "start_stamp": "2024-09-11T08:00:43.954031+0000", 2024-09-11T08:03:38.621 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "gid": 24395, 2024-09-11T08:03:38.621 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "addr": "172.21.15.102:0/502152468", 2024-09-11T08:03:38.621 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "metadata": { 2024-09-11T08:03:38.621 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "arch": "x86_64", 2024-09-11T08:03:38.621 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "ceph_release": "quincy", 2024-09-11T08:03:38.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1820385928' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:38.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[41371]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:38.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[41371]: from='client.34895 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:38.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1820385928' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:38.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:38.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:38.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[41371]: from='client.34895 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:38.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1820385928' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:38.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:38.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[41371]: from='client.34895 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:38.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:03:38.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[41371]: osdmap e90: 8 total, 8 up, 8 in 2024-09-11T08:03:38.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2211808202' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi102-80356-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:38.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2969981406' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi102-80338-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:38.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[41371]: from='client.35311 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi102-80356-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:38.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[41371]: from='client.35389 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi102-80338-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:38.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1820385928' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:38.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[29849]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:38.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[29849]: from='client.34895 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:38.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1820385928' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:38.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:38.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:38.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[29849]: from='client.34895 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:38.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1820385928' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:38.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:38.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[29849]: from='client.34895 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:38.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:03:38.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[29849]: osdmap e90: 8 total, 8 up, 8 in 2024-09-11T08:03:38.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2211808202' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi102-80356-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:38.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2969981406' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi102-80338-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:38.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[29849]: from='client.35311 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi102-80356-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:38.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:38 smithi102 ceph-mon[29849]: from='client.35389 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi102-80338-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:38.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1820385928' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:38.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:38 smithi146 ceph-mon[33158]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:38.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:38 smithi146 ceph-mon[33158]: from='client.34895 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:38.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1820385928' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:38.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:38 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:03:38.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:38 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:03:38.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:38 smithi146 ceph-mon[33158]: from='client.34895 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:38.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1820385928' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:38.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:38.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:38 smithi146 ceph-mon[33158]: from='client.34895 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:38.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:38 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:03:38.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:38 smithi146 ceph-mon[33158]: osdmap e90: 8 total, 8 up, 8 in 2024-09-11T08:03:38.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2211808202' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi102-80356-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:38.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2969981406' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi102-80338-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:38.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:38 smithi146 ceph-mon[33158]: from='client.35311 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi102-80356-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:38.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:38 smithi146 ceph-mon[33158]: from='client.35389 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi102-80338-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:39.013 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064 api_c_write_operations: Running main() from gmock_main.cc 2024-09-11T08:03:39.013 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-09-11T08:03:39.013 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-09-11T08:03:39.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-09-11T08:03:39.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-09-11T08:03:39.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-09-11T08:03:39.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-09-11T08:03:39.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: 2024-09-11T08:03:39.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-09-11T08:03:39.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-09-11T08:03:39.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (8624 ms) 2024-09-11T08:03:39.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-09-11T08:03:39.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (4034 ms) 2024-09-11T08:03:39.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-09-11T08:03:39.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (3295 ms) 2024-09-11T08:03:39.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-09-11T08:03:39.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (3530 ms) 2024-09-11T08:03:39.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-09-11T08:03:39.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (3001 ms) 2024-09-11T08:03:39.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-09-11T08:03:39.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (4094 ms) 2024-09-11T08:03:39.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-09-11T08:03:39.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (4082 ms) 2024-09-11T08:03:39.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (30660 ms total) 2024-09-11T08:03:39.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: 2024-09-11T08:03:39.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-09-11T08:03:39.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (30660 ms total) 2024-09-11T08:03:39.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-09-11T08:03:39.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:39 smithi102 ceph-mon[29849]: pgmap v104: 552 pgs: 160 unknown, 392 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:03:39.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:39 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1548342967' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:03:39.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:39 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:03:39.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:39 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:39.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:39 smithi102 ceph-mon[29849]: from='client.34895 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:39.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:39 smithi102 ceph-mon[29849]: from='client.35311 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi102-80356-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:39.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:39 smithi102 ceph-mon[29849]: from='client.35389 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi102-80338-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:39.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:39 smithi102 ceph-mon[29849]: osdmap e91: 8 total, 8 up, 8 in 2024-09-11T08:03:39.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:39 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1820385928' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi102-80709-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:39.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:39 smithi102 ceph-mon[29849]: from='client.34895 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi102-80709-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:39.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:39 smithi102 ceph-mon[41371]: pgmap v104: 552 pgs: 160 unknown, 392 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:03:39.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:39 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1548342967' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:03:39.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:39 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:03:39.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:39 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:39.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:39 smithi102 ceph-mon[41371]: from='client.34895 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:39.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:39 smithi102 ceph-mon[41371]: from='client.35311 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi102-80356-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:39.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:39 smithi102 ceph-mon[41371]: from='client.35389 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi102-80338-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:39.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:39 smithi102 ceph-mon[41371]: osdmap e91: 8 total, 8 up, 8 in 2024-09-11T08:03:39.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:39 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1820385928' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi102-80709-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:39.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:39 smithi102 ceph-mon[41371]: from='client.34895 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi102-80709-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:39.683 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:03:39 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:03:39.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:39 smithi146 ceph-mon[33158]: pgmap v104: 552 pgs: 160 unknown, 392 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:03:39.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:39 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1548342967' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:03:39.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:39 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:03:39.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:39 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:39.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:39 smithi146 ceph-mon[33158]: from='client.34895 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:39.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:39 smithi146 ceph-mon[33158]: from='client.35311 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi102-80356-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:39.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:39 smithi146 ceph-mon[33158]: from='client.35389 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi102-80338-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:39.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:39 smithi146 ceph-mon[33158]: osdmap e91: 8 total, 8 up, 8 in 2024-09-11T08:03:39.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:39 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1820385928' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi102-80709-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:39.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:39 smithi146 ceph-mon[33158]: from='client.34895 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi102-80709-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:40.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:03:39 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:03:39] "GET /metrics HTTP/1.1" 200 251805 "" "Prometheus/2.43.0" 2024-09-11T08:03:41.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:40 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:03:41.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:41.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:41.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:40 smithi102 ceph-mon[41371]: osdmap e92: 8 total, 8 up, 8 in 2024-09-11T08:03:41.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:40 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:41.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:40 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:03:41.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:41.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:41.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:40 smithi102 ceph-mon[29849]: osdmap e92: 8 total, 8 up, 8 in 2024-09-11T08:03:41.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:40 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:41.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:40 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:03:41.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:41.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:41.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:40 smithi146 ceph-mon[33158]: osdmap e92: 8 total, 8 up, 8 in 2024-09-11T08:03:41.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:40 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:41 smithi102 ceph-mon[41371]: pgmap v107: 392 pgs: 28 creating+peering, 4 unknown, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:03:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:41 smithi102 ceph-mon[41371]: from='client.34895 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi102-80709-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]': finished 2024-09-11T08:03:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:41 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2373809287' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi102-80338-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1284054306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:42.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:41 smithi102 ceph-mon[41371]: osdmap e93: 8 total, 8 up, 8 in 2024-09-11T08:03:42.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:41 smithi102 ceph-mon[41371]: from='client.39262 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi102-80338-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:42.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:41 smithi102 ceph-mon[41371]: from='client.38881 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:42.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:41 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:42.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:41 smithi102 ceph-mon[29849]: pgmap v107: 392 pgs: 28 creating+peering, 4 unknown, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:03:42.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:42.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:41 smithi102 ceph-mon[29849]: from='client.34895 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi102-80709-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]': finished 2024-09-11T08:03:42.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:41 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:42.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2373809287' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi102-80338-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:42.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1284054306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:42.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:42.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:41 smithi102 ceph-mon[29849]: osdmap e93: 8 total, 8 up, 8 in 2024-09-11T08:03:42.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:41 smithi102 ceph-mon[29849]: from='client.39262 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi102-80338-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:42.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:41 smithi102 ceph-mon[29849]: from='client.38881 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:42.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:41 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:42.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:41 smithi146 ceph-mon[33158]: pgmap v107: 392 pgs: 28 creating+peering, 4 unknown, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:03:42.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:42.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:41 smithi146 ceph-mon[33158]: from='client.34895 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi102-80709-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]': finished 2024-09-11T08:03:42.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:41 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:42.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2373809287' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi102-80338-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:42.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1284054306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:42.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:42.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:41 smithi146 ceph-mon[33158]: osdmap e93: 8 total, 8 up, 8 in 2024-09-11T08:03:42.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:41 smithi146 ceph-mon[33158]: from='client.39262 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi102-80338-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:42.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:41 smithi146 ceph-mon[33158]: from='client.38881 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:42.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:41 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:43.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:43.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:42 smithi102 ceph-mon[41371]: from='client.39262 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi102-80338-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:43.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:42 smithi102 ceph-mon[41371]: from='client.38881 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:43.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:42 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:03:43.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:42 smithi102 ceph-mon[41371]: osdmap e94: 8 total, 8 up, 8 in 2024-09-11T08:03:43.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1483915912' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi102-80356-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:43.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-9"}]: dispatch 2024-09-11T08:03:43.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:42 smithi102 ceph-mon[41371]: from='client.39268 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi102-80356-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:43.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:42 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-9"}]: dispatch 2024-09-11T08:03:43.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4"}]: dispatch 2024-09-11T08:03:43.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:42 smithi102 ceph-mon[41371]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4"}]: dispatch 2024-09-11T08:03:43.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:43.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:42 smithi102 ceph-mon[29849]: from='client.39262 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi102-80338-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:43.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:42 smithi102 ceph-mon[29849]: from='client.38881 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:43.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:42 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:03:43.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:42 smithi102 ceph-mon[29849]: osdmap e94: 8 total, 8 up, 8 in 2024-09-11T08:03:43.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1483915912' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi102-80356-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:43.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-9"}]: dispatch 2024-09-11T08:03:43.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:42 smithi102 ceph-mon[29849]: from='client.39268 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi102-80356-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:43.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:42 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-9"}]: dispatch 2024-09-11T08:03:43.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4"}]: dispatch 2024-09-11T08:03:43.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:42 smithi102 ceph-mon[29849]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4"}]: dispatch 2024-09-11T08:03:43.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:43.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:42 smithi146 ceph-mon[33158]: from='client.39262 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi102-80338-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:43.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:42 smithi146 ceph-mon[33158]: from='client.38881 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:43.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:42 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:03:43.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:42 smithi146 ceph-mon[33158]: osdmap e94: 8 total, 8 up, 8 in 2024-09-11T08:03:43.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1483915912' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi102-80356-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:43.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-9"}]: dispatch 2024-09-11T08:03:43.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:42 smithi146 ceph-mon[33158]: from='client.39268 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi102-80356-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:43.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:42 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-9"}]: dispatch 2024-09-11T08:03:43.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4"}]: dispatch 2024-09-11T08:03:43.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:42 smithi146 ceph-mon[33158]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4"}]: dispatch 2024-09-11T08:03:43.768 INFO:tasks.workunit.client.0.smithi102.stdout:e50742bb1c39b9) quincy (stable)", 2024-09-11T08:03:43.768 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "ceph_version_short": "17.2.7-1623-gd0e6828a", 2024-09-11T08:03:43.768 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "container_hostname": "smithi102", 2024-09-11T08:03:43.768 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40", 2024-09-11T08:03:43.768 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-09-11T08:03:43.769 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "distro": "centos", 2024-09-11T08:03:43.769 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-09-11T08:03:43.769 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "distro_version": "9", 2024-09-11T08:03:43.769 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-09-11T08:03:43.769 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "frontend_type#0": "beast", 2024-09-11T08:03:43.769 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "hostname": "smithi102", 2024-09-11T08:03:43.769 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "id": "foo.a", 2024-09-11T08:03:43.769 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024", 2024-09-11T08:03:43.769 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "kernel_version": "5.14.0-505.el9.x86_64", 2024-09-11T08:03:43.769 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "mem_swap_kb": "0", 2024-09-11T08:03:43.769 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "mem_total_kb": "32487292", 2024-09-11T08:03:43.769 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "num_handles": "1", 2024-09-11T08:03:43.769 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "os": "Linux", 2024-09-11T08:03:43.769 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "pid": "7", 2024-09-11T08:03:43.770 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "realm_id": "", 2024-09-11T08:03:43.770 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "realm_name": "", 2024-09-11T08:03:43.770 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "zone_id": "a569d787-a4cc-4dc1-b647-64d02e649f29", 2024-09-11T08:03:43.770 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "zone_name": "default", 2024-09-11T08:03:43.770 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "zonegroup_id": "06b663fd-a407-41e5-8577-d056304f39f3", 2024-09-11T08:03:43.770 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "zonegroup_name": "default" 2024-09-11T08:03:43.770 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: }, 2024-09-11T08:03:43.770 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: "task_status": {} 2024-09-11T08:03:43.770 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: } 2024-09-11T08:03:43.770 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: } 2024-09-11T08:03:43.770 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: } 2024-09-11T08:03:43.770 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: } 2024-09-11T08:03:43.770 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: } 2024-09-11T08:03:43.770 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: 2024-09-11T08:03:43.771 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: attempt 1 of 20 2024-09-11T08:03:43.771 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (10287 ms) 2024-09-11T08:03:43.771 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (35412 ms total) 2024-09-11T08:03:43.771 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: 2024-09-11T08:03:43.771 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: [----------] Global test environment tear-down 2024-09-11T08:03:43.771 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (35412 ms total) 2024-09-11T08:03:43.771 INFO:tasks.workunit.client.0.smithi102.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-09-11T08:03:44.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:43 smithi102 ceph-mon[29849]: pgmap v110: 496 pgs: 28 creating+peering, 108 unknown, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-11T08:03:44.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:43 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:44.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:43 smithi102 ceph-mon[29849]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:44.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:43 smithi102 ceph-mon[29849]: from='client.39268 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi102-80356-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:44.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:43 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-9"}]': finished 2024-09-11T08:03:44.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:43 smithi102 ceph-mon[29849]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4"}]': finished 2024-09-11T08:03:44.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:43 smithi102 ceph-mon[29849]: osdmap e95: 8 total, 8 up, 8 in 2024-09-11T08:03:44.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:43 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6"}]: dispatch 2024-09-11T08:03:44.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:43 smithi102 ceph-mon[29849]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6"}]: dispatch 2024-09-11T08:03:44.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:43 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-9", "mode": "writeback"}]: dispatch 2024-09-11T08:03:44.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:43 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-9", "mode": "writeback"}]: dispatch 2024-09-11T08:03:44.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:43 smithi102 ceph-mon[41371]: pgmap v110: 496 pgs: 28 creating+peering, 108 unknown, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-11T08:03:44.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:43 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:44.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:43 smithi102 ceph-mon[41371]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:44.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:43 smithi102 ceph-mon[41371]: from='client.39268 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi102-80356-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:44.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:43 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-9"}]': finished 2024-09-11T08:03:44.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:43 smithi102 ceph-mon[41371]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4"}]': finished 2024-09-11T08:03:44.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:43 smithi102 ceph-mon[41371]: osdmap e95: 8 total, 8 up, 8 in 2024-09-11T08:03:44.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:43 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6"}]: dispatch 2024-09-11T08:03:44.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:43 smithi102 ceph-mon[41371]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6"}]: dispatch 2024-09-11T08:03:44.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:43 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-9", "mode": "writeback"}]: dispatch 2024-09-11T08:03:44.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:43 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-9", "mode": "writeback"}]: dispatch 2024-09-11T08:03:44.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:43 smithi146 ceph-mon[33158]: pgmap v110: 496 pgs: 28 creating+peering, 108 unknown, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-11T08:03:44.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:43 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:44.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:43 smithi146 ceph-mon[33158]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:44.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:43 smithi146 ceph-mon[33158]: from='client.39268 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi102-80356-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:44.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:43 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-9"}]': finished 2024-09-11T08:03:44.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:43 smithi146 ceph-mon[33158]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4"}]': finished 2024-09-11T08:03:44.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:43 smithi146 ceph-mon[33158]: osdmap e95: 8 total, 8 up, 8 in 2024-09-11T08:03:44.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:43 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6"}]: dispatch 2024-09-11T08:03:44.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:43 smithi146 ceph-mon[33158]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6"}]: dispatch 2024-09-11T08:03:44.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:43 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-9", "mode": "writeback"}]: dispatch 2024-09-11T08:03:44.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:43 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-9", "mode": "writeback"}]: dispatch 2024-09-11T08:03:44.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:03:44 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:03:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:03:44.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:44 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:44.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:44 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/782206946' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:03:44.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:44 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:03:44.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:44 smithi146 ceph-mon[33158]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6"}]': finished 2024-09-11T08:03:44.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:44 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-9", "mode": "writeback"}]': finished 2024-09-11T08:03:44.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:44 smithi146 ceph-mon[33158]: osdmap e96: 8 total, 8 up, 8 in 2024-09-11T08:03:44.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:44 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:03:44.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:44 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1963134409' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi102-80338-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:44.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:44 smithi146 ceph-mon[33158]: from='client.42313 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi102-80338-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:44.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:44 smithi146 ceph-mon[33158]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:03:44.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:44 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:03:44.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:44 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:03:44.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:44 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:03:44.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:44 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:03:45.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:45.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/782206946' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:03:45.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:03:45.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[41371]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6"}]': finished 2024-09-11T08:03:45.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-9", "mode": "writeback"}]': finished 2024-09-11T08:03:45.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[41371]: osdmap e96: 8 total, 8 up, 8 in 2024-09-11T08:03:45.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:03:45.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1963134409' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi102-80338-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:45.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[41371]: from='client.42313 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi102-80338-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:45.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[41371]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:03:45.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:03:45.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:03:45.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:03:45.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:03:45.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:45.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/782206946' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:03:45.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.102:0/1772904340' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-11T08:03:45.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[29849]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6"}]': finished 2024-09-11T08:03:45.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-9", "mode": "writeback"}]': finished 2024-09-11T08:03:45.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[29849]: osdmap e96: 8 total, 8 up, 8 in 2024-09-11T08:03:45.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/383087494' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:03:45.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1963134409' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi102-80338-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:45.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[29849]: from='client.42313 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi102-80338-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:45.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[29849]: from='client.25187 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:03:45.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:03:45.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:03:45.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:03:45.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:44 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:03:45.933 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: flushed 2024-09-11T08:03:45.933 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3007 ms) 2024-09-11T08:03:45.933 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-09-11T08:03:45.933 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: trying... 2024-09-11T08:03:45.933 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: handle_notify cookie 94152800501440 notify_id 365072220165 notifier_gid 25187 2024-09-11T08:03:45.933 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: timed out 2024-09-11T08:03:45.933 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: flushing 2024-09-11T08:03:45.933 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: flushed 2024-09-11T08:03:45.934 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3016 ms) 2024-09-11T08:03:45.934 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-09-11T08:03:45.934 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: List watches 2024-09-11T08:03:45.934 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: notify2 2024-09-11T08:03:45.934 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: handle_notify cookie 94152800490880 notify_id 377957122050 notifier_gid 25187 2024-09-11T08:03:45.934 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: notify2 done 2024-09-11T08:03:45.934 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: watch_check 2024-09-11T08:03:45.934 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: unwatch2 2024-09-11T08:03:45.934 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: flushing 2024-09-11T08:03:45.934 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: done 2024-09-11T08:03:45.934 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3009 ms) 2024-09-11T08:03:45.934 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-09-11T08:03:45.934 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: List watches 2024-09-11T08:03:45.934 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: notify2 2024-09-11T08:03:45.935 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: handle_notify cookie 94152800490880 notify_id 390842023939 notifier_gid 25187 2024-09-11T08:03:45.935 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: notify2 done 2024-09-11T08:03:45.935 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: watch_check 2024-09-11T08:03:45.935 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: unwatch2 2024-09-11T08:03:45.935 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: flushing 2024-09-11T08:03:45.935 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: done 2024-09-11T08:03:45.935 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3012 ms) 2024-09-11T08:03:45.935 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (16778 ms total) 2024-09-11T08:03:45.935 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: 2024-09-11T08:03:45.935 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-09-11T08:03:45.935 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (37658 ms total) 2024-09-11T08:03:45.935 INFO:tasks.workunit.client.0.smithi102.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-09-11T08:03:46.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:45 smithi102 ceph-mon[29849]: pgmap v113: 464 pgs: 28 creating+peering, 76 unknown, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:03:46.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:45 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:46.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:45 smithi102 ceph-mon[29849]: from='client.42313 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi102-80338-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:46.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:45 smithi102 ceph-mon[29849]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:03:46.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:45 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:03:46.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:45 smithi102 ceph-mon[29849]: osdmap e97: 8 total, 8 up, 8 in 2024-09-11T08:03:46.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:45 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-9"}]: dispatch 2024-09-11T08:03:46.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:45 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-9"}]: dispatch 2024-09-11T08:03:46.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:45 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2914287788' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi102-80356-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:46.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:45 smithi102 ceph-mon[29849]: from='client.43441 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi102-80356-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:46.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:45 smithi102 ceph-mon[41371]: pgmap v113: 464 pgs: 28 creating+peering, 76 unknown, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:03:46.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:45 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:46.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:45 smithi102 ceph-mon[41371]: from='client.42313 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi102-80338-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:46.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:45 smithi102 ceph-mon[41371]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:03:46.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:45 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:03:46.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:45 smithi102 ceph-mon[41371]: osdmap e97: 8 total, 8 up, 8 in 2024-09-11T08:03:46.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:45 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-9"}]: dispatch 2024-09-11T08:03:46.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:45 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-9"}]: dispatch 2024-09-11T08:03:46.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:45 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2914287788' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi102-80356-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:46.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:45 smithi102 ceph-mon[41371]: from='client.43441 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi102-80356-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:46.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:45 smithi146 ceph-mon[33158]: pgmap v113: 464 pgs: 28 creating+peering, 76 unknown, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:03:46.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:45 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:46.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:45 smithi146 ceph-mon[33158]: from='client.42313 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi102-80338-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:46.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:45 smithi146 ceph-mon[33158]: from='client.25187 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi102-80785-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:03:46.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:45 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:03:46.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:45 smithi146 ceph-mon[33158]: osdmap e97: 8 total, 8 up, 8 in 2024-09-11T08:03:46.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:45 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-9"}]: dispatch 2024-09-11T08:03:46.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:45 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-9"}]: dispatch 2024-09-11T08:03:46.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:45 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2914287788' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi102-80356-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:46.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:45 smithi146 ceph-mon[33158]: from='client.43441 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi102-80356-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:46.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:46 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:46.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:46 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:03:46.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:46 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-9"}]': finished 2024-09-11T08:03:46.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:46 smithi146 ceph-mon[33158]: from='client.43441 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi102-80356-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:46.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:46 smithi146 ceph-mon[33158]: osdmap e98: 8 total, 8 up, 8 in 2024-09-11T08:03:46.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:46 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/155065641' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:46.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:46 smithi146 ceph-mon[33158]: from='client.43553 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:47.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:47.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:46 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:03:47.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:46 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-9"}]': finished 2024-09-11T08:03:47.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:46 smithi102 ceph-mon[29849]: from='client.43441 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi102-80356-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:47.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:46 smithi102 ceph-mon[29849]: osdmap e98: 8 total, 8 up, 8 in 2024-09-11T08:03:47.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/155065641' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:47.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:46 smithi102 ceph-mon[29849]: from='client.43553 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:47.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:46 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:47.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:46 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:03:47.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:46 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-9"}]': finished 2024-09-11T08:03:47.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:46 smithi102 ceph-mon[41371]: from='client.43441 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi102-80356-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:47.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:46 smithi102 ceph-mon[41371]: osdmap e98: 8 total, 8 up, 8 in 2024-09-11T08:03:47.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:46 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/155065641' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:47.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:46 smithi102 ceph-mon[41371]: from='client.43553 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:47.908 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-09-11T08:03:47.908 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [----------] Global test environment set-up. 2024-09-11T08:03:47.908 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-09-11T08:03:47.908 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-09-11T08:03:47.908 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-09-11T08:03:47.908 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-09-11T08:03:47.908 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: 2024-09-11T08:03:47.908 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-09-11T08:03:47.908 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-09-11T08:03:47.909 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-09-11T08:03:47.909 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: 2024-09-11T08:03:08.247+0000 7f5f34cb3600 -1 failed for service _ceph-mon._tcp 2024-09-11T08:03:47.909 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: 2024-09-11T08:03:08.247+0000 7f5f34cb3600 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-09-11T08:03:47.909 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (44 ms) 2024-09-11T08:03:47.909 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-09-11T08:03:47.909 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (5016 ms) 2024-09-11T08:03:47.909 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (5060 ms total) 2024-09-11T08:03:47.909 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: 2024-09-11T08:03:47.909 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-09-11T08:03:47.909 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-09-11T08:03:47.909 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: started 0x5612f7f9d2d0 2024-09-11T08:03:47.909 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: started 0x5612f7fa1ad0 2024-09-11T08:03:47.910 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: started 2 aios 2024-09-11T08:03:47.910 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: waiting 0x5612f7f9d2d0 2024-09-11T08:03:47.910 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: waiting 0x5612f7fa1ad0 2024-09-11T08:03:47.910 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: done. 2024-09-11T08:03:47.910 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (7037 ms) 2024-09-11T08:03:47.910 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (7037 ms total) 2024-09-11T08:03:47.910 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: 2024-09-11T08:03:47.910 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-09-11T08:03:47.910 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-09-11T08:03:47.910 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-09-11T08:03:47.910 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-09-11T08:03:47.910 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ OK ] LibRadosMisc.Exec (42 ms) 2024-09-11T08:03:47.910 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-09-11T08:03:47.911 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (5 ms) 2024-09-11T08:03:47.911 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-09-11T08:03:47.911 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (4 ms) 2024-09-11T08:03:47.911 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-09-11T08:03:47.911 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ OK ] LibRadosMisc.Applications (5512 ms) 2024-09-11T08:03:47.911 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-09-11T08:03:47.911 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-09-11T08:03:47.911 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-09-11T08:03:47.911 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-09-11T08:03:47.911 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-09-11T08:03:47.911 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (19465 ms) 2024-09-11T08:03:47.911 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [----------] 8 tests from LibRadosMisc (25028 ms total) 2024-09-11T08:03:47.911 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: 2024-09-11T08:03:47.911 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [----------] Global test environment tear-down 2024-09-11T08:03:47.912 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (39695 ms total) 2024-09-11T08:03:47.912 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc: [ PASSED ] 12 tests. 2024-09-11T08:03:48.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:47 smithi102 ceph-mon[41371]: pgmap v116: 428 pgs: 32 creating+peering, 32 unknown, 364 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 767 B/s wr, 1 op/s 2024-09-11T08:03:48.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:48.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:47 smithi102 ceph-mon[41371]: from='client.43553 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:48.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:47 smithi102 ceph-mon[41371]: osdmap e99: 8 total, 8 up, 8 in 2024-09-11T08:03:48.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1208880251' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi102-80338-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:48.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:47 smithi102 ceph-mon[41371]: from='client.45829 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi102-80338-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:48.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:47 smithi102 ceph-mon[29849]: pgmap v116: 428 pgs: 32 creating+peering, 32 unknown, 364 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 767 B/s wr, 1 op/s 2024-09-11T08:03:48.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:48.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:47 smithi102 ceph-mon[29849]: from='client.43553 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:48.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:47 smithi102 ceph-mon[29849]: osdmap e99: 8 total, 8 up, 8 in 2024-09-11T08:03:48.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1208880251' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi102-80338-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:48.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:47 smithi102 ceph-mon[29849]: from='client.45829 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi102-80338-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:47 smithi146 ceph-mon[33158]: pgmap v116: 428 pgs: 32 creating+peering, 32 unknown, 364 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 767 B/s wr, 1 op/s 2024-09-11T08:03:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:47 smithi146 ceph-mon[33158]: from='client.43553 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80700-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:47 smithi146 ceph-mon[33158]: osdmap e99: 8 total, 8 up, 8 in 2024-09-11T08:03:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1208880251' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi102-80338-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:47 smithi146 ceph-mon[33158]: from='client.45829 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi102-80338-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:49.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:49.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.0"}]: dispatch 2024-09-11T08:03:49.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.0"}]: dispatch 2024-09-11T08:03:49.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.1"}]: dispatch 2024-09-11T08:03:49.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.1"}]: dispatch 2024-09-11T08:03:49.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.2"}]: dispatch 2024-09-11T08:03:49.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.2"}]: dispatch 2024-09-11T08:03:49.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.3"}]: dispatch 2024-09-11T08:03:49.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.3"}]: dispatch 2024-09-11T08:03:49.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.4"}]: dispatch 2024-09-11T08:03:49.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.4"}]: dispatch 2024-09-11T08:03:49.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.5"}]: dispatch 2024-09-11T08:03:49.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.5"}]: dispatch 2024-09-11T08:03:49.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.6"}]: dispatch 2024-09-11T08:03:49.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.6"}]: dispatch 2024-09-11T08:03:49.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.7"}]: dispatch 2024-09-11T08:03:49.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.7"}]: dispatch 2024-09-11T08:03:49.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.8"}]: dispatch 2024-09-11T08:03:49.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.8"}]: dispatch 2024-09-11T08:03:49.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.9"}]: dispatch 2024-09-11T08:03:49.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.9"}]: dispatch 2024-09-11T08:03:49.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='client.45829 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi102-80338-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:49.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: osdmap e100: 8 total, 8 up, 8 in 2024-09-11T08:03:49.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/155065641' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80700-10", "tierpool":"test-rados-api-smithi102-80700-10-cache", "force_nonempty":""}]: dispatch 2024-09-11T08:03:49.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:49.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.0"}]: dispatch 2024-09-11T08:03:49.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.0"}]: dispatch 2024-09-11T08:03:49.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.1"}]: dispatch 2024-09-11T08:03:49.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.1"}]: dispatch 2024-09-11T08:03:49.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.2"}]: dispatch 2024-09-11T08:03:49.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.2"}]: dispatch 2024-09-11T08:03:49.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.3"}]: dispatch 2024-09-11T08:03:49.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.3"}]: dispatch 2024-09-11T08:03:49.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.4"}]: dispatch 2024-09-11T08:03:49.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.4"}]: dispatch 2024-09-11T08:03:49.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.5"}]: dispatch 2024-09-11T08:03:49.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.5"}]: dispatch 2024-09-11T08:03:49.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.6"}]: dispatch 2024-09-11T08:03:49.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.6"}]: dispatch 2024-09-11T08:03:49.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.7"}]: dispatch 2024-09-11T08:03:49.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.7"}]: dispatch 2024-09-11T08:03:49.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.8"}]: dispatch 2024-09-11T08:03:49.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.8"}]: dispatch 2024-09-11T08:03:49.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.9"}]: dispatch 2024-09-11T08:03:49.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.9"}]: dispatch 2024-09-11T08:03:49.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='client.45829 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi102-80338-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:49.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: osdmap e100: 8 total, 8 up, 8 in 2024-09-11T08:03:49.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/155065641' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80700-10", "tierpool":"test-rados-api-smithi102-80700-10-cache", "force_nonempty":""}]: dispatch 2024-09-11T08:03:49.150 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3684416418' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi102-80356-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:49.150 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:49.150 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='client.43553 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80700-10", "tierpool":"test-rados-api-smithi102-80700-10-cache", "force_nonempty":""}]: dispatch 2024-09-11T08:03:49.150 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='client.47074 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi102-80356-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:49.150 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:49.150 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: 22.2 deep-scrub starts 2024-09-11T08:03:49.150 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: 22.1 deep-scrub starts 2024-09-11T08:03:49.150 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: 22.1 deep-scrub ok 2024-09-11T08:03:49.150 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: 22.2 deep-scrub ok 2024-09-11T08:03:49.150 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: 22.7 deep-scrub starts 2024-09-11T08:03:49.150 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: 22.7 deep-scrub ok 2024-09-11T08:03:49.151 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:49.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3684416418' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi102-80356-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:49.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:49.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='client.43553 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80700-10", "tierpool":"test-rados-api-smithi102-80700-10-cache", "force_nonempty":""}]: dispatch 2024-09-11T08:03:49.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='client.47074 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi102-80356-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:49.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:49.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: 22.2 deep-scrub starts 2024-09-11T08:03:49.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: 22.1 deep-scrub starts 2024-09-11T08:03:49.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: 22.1 deep-scrub ok 2024-09-11T08:03:49.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: 22.2 deep-scrub ok 2024-09-11T08:03:49.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: 22.7 deep-scrub starts 2024-09-11T08:03:49.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: 22.7 deep-scrub ok 2024-09-11T08:03:49.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:49.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:49.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.0"}]: dispatch 2024-09-11T08:03:49.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.0"}]: dispatch 2024-09-11T08:03:49.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.1"}]: dispatch 2024-09-11T08:03:49.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.1"}]: dispatch 2024-09-11T08:03:49.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.2"}]: dispatch 2024-09-11T08:03:49.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.2"}]: dispatch 2024-09-11T08:03:49.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.3"}]: dispatch 2024-09-11T08:03:49.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.3"}]: dispatch 2024-09-11T08:03:49.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.4"}]: dispatch 2024-09-11T08:03:49.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.4"}]: dispatch 2024-09-11T08:03:49.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.5"}]: dispatch 2024-09-11T08:03:49.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.5"}]: dispatch 2024-09-11T08:03:49.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.6"}]: dispatch 2024-09-11T08:03:49.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.6"}]: dispatch 2024-09-11T08:03:49.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.7"}]: dispatch 2024-09-11T08:03:49.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.7"}]: dispatch 2024-09-11T08:03:49.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.8"}]: dispatch 2024-09-11T08:03:49.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.8"}]: dispatch 2024-09-11T08:03:49.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.9"}]: dispatch 2024-09-11T08:03:49.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "22.9"}]: dispatch 2024-09-11T08:03:49.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='client.45829 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi102-80338-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:49.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: osdmap e100: 8 total, 8 up, 8 in 2024-09-11T08:03:49.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/155065641' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80700-10", "tierpool":"test-rados-api-smithi102-80700-10-cache", "force_nonempty":""}]: dispatch 2024-09-11T08:03:49.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3684416418' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi102-80356-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:49.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:49.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='client.43553 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80700-10", "tierpool":"test-rados-api-smithi102-80700-10-cache", "force_nonempty":""}]: dispatch 2024-09-11T08:03:49.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='client.47074 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi102-80356-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:49.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:49.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: 22.2 deep-scrub starts 2024-09-11T08:03:49.186 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: 22.1 deep-scrub starts 2024-09-11T08:03:49.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: 22.1 deep-scrub ok 2024-09-11T08:03:49.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: 22.2 deep-scrub ok 2024-09-11T08:03:49.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: 22.7 deep-scrub starts 2024-09-11T08:03:49.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: 22.7 deep-scrub ok 2024-09-11T08:03:49.187 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:49.789 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:03:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:03:50.046 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:03:49] "GET /metrics HTTP/1.1" 200 249588 "" "Prometheus/2.43.0" 2024-09-11T08:03:50.047 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[29849]: pgmap v119: 460 pgs: 160 unknown, 300 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:03:50.047 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[29849]: 22.0 deep-scrub starts 2024-09-11T08:03:50.047 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[29849]: 22.0 deep-scrub ok 2024-09-11T08:03:50.047 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[29849]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:50.047 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[29849]: from='client.43553 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80700-10", "tierpool":"test-rados-api-smithi102-80700-10-cache", "force_nonempty":""}]': finished 2024-09-11T08:03:50.047 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[29849]: from='client.47074 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi102-80356-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:50.047 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:50.048 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[29849]: osdmap e101: 8 total, 8 up, 8 in 2024-09-11T08:03:50.048 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/155065641' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80700-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:50.048 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[29849]: from='client.43553 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80700-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:50.048 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[29849]: 22.9 deep-scrub starts 2024-09-11T08:03:50.048 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[29849]: 22.9 deep-scrub ok 2024-09-11T08:03:50.048 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:50.048 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[41371]: pgmap v119: 460 pgs: 160 unknown, 300 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:03:50.048 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[41371]: 22.0 deep-scrub starts 2024-09-11T08:03:50.048 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[41371]: 22.0 deep-scrub ok 2024-09-11T08:03:50.048 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[41371]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:50.048 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[41371]: from='client.43553 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80700-10", "tierpool":"test-rados-api-smithi102-80700-10-cache", "force_nonempty":""}]': finished 2024-09-11T08:03:50.048 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[41371]: from='client.47074 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi102-80356-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:50.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:50.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[41371]: osdmap e101: 8 total, 8 up, 8 in 2024-09-11T08:03:50.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/155065641' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80700-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:50.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[41371]: from='client.43553 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80700-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:50.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[41371]: 22.9 deep-scrub starts 2024-09-11T08:03:50.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[41371]: 22.9 deep-scrub ok 2024-09-11T08:03:50.049 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:49 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:49 smithi146 ceph-mon[33158]: pgmap v119: 460 pgs: 160 unknown, 300 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:03:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:49 smithi146 ceph-mon[33158]: 22.0 deep-scrub starts 2024-09-11T08:03:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:49 smithi146 ceph-mon[33158]: 22.0 deep-scrub ok 2024-09-11T08:03:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:49 smithi146 ceph-mon[33158]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:49 smithi146 ceph-mon[33158]: from='client.43553 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80700-10", "tierpool":"test-rados-api-smithi102-80700-10-cache", "force_nonempty":""}]': finished 2024-09-11T08:03:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:49 smithi146 ceph-mon[33158]: from='client.47074 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi102-80356-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:49 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:49 smithi146 ceph-mon[33158]: osdmap e101: 8 total, 8 up, 8 in 2024-09-11T08:03:50.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:49 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/155065641' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80700-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:50.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:49 smithi146 ceph-mon[33158]: from='client.43553 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80700-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:50.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:49 smithi146 ceph-mon[33158]: 22.9 deep-scrub starts 2024-09-11T08:03:50.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:49 smithi146 ceph-mon[33158]: 22.9 deep-scrub ok 2024-09-11T08:03:50.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:49 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:51.062 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:03:51.062 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[41371]: from='client.43553 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80700-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:51.063 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[41371]: osdmap e102: 8 total, 8 up, 8 in 2024-09-11T08:03:51.063 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/155065641' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80700-10", "tierpool":"test-rados-api-smithi102-80700-10-cache"}]: dispatch 2024-09-11T08:03:51.063 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[41371]: from='client.43553 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80700-10", "tierpool":"test-rados-api-smithi102-80700-10-cache"}]: dispatch 2024-09-11T08:03:51.063 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3894037500' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi102-80338-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:51.063 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[41371]: from='client.46454 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi102-80338-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:51.063 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:51.063 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:51.063 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[41371]: 22.5 deep-scrub starts 2024-09-11T08:03:51.063 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[41371]: 22.5 deep-scrub ok 2024-09-11T08:03:51.063 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:51.063 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:03:51.064 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[29849]: from='client.43553 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80700-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:51.064 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[29849]: osdmap e102: 8 total, 8 up, 8 in 2024-09-11T08:03:51.064 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/155065641' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80700-10", "tierpool":"test-rados-api-smithi102-80700-10-cache"}]: dispatch 2024-09-11T08:03:51.064 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[29849]: from='client.43553 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80700-10", "tierpool":"test-rados-api-smithi102-80700-10-cache"}]: dispatch 2024-09-11T08:03:51.064 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3894037500' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi102-80338-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:51.064 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[29849]: from='client.46454 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi102-80338-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:51.064 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:51.064 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:51.064 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[29849]: 22.5 deep-scrub starts 2024-09-11T08:03:51.064 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[29849]: 22.5 deep-scrub ok 2024-09-11T08:03:51.064 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:50 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:51.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:50 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:03:51.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:50 smithi146 ceph-mon[33158]: from='client.43553 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80700-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:51.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:50 smithi146 ceph-mon[33158]: osdmap e102: 8 total, 8 up, 8 in 2024-09-11T08:03:51.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:50 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/155065641' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80700-10", "tierpool":"test-rados-api-smithi102-80700-10-cache"}]: dispatch 2024-09-11T08:03:51.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:50 smithi146 ceph-mon[33158]: from='client.43553 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80700-10", "tierpool":"test-rados-api-smithi102-80700-10-cache"}]: dispatch 2024-09-11T08:03:51.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:50 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3894037500' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi102-80338-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:51.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:50 smithi146 ceph-mon[33158]: from='client.46454 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi102-80338-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:51.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:50 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:51.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:50 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:03:51.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:50 smithi146 ceph-mon[33158]: 22.5 deep-scrub starts 2024-09-11T08:03:51.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:50 smithi146 ceph-mon[33158]: 22.5 deep-scrub ok 2024-09-11T08:03:51.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:50 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:51.792 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[29849]: 22.6 deep-scrub starts 2024-09-11T08:03:51.792 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[29849]: 22.6 deep-scrub ok 2024-09-11T08:03:51.792 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[29849]: pgmap v122: 428 pgs: 32 creating+peering, 396 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 18 MiB/s wr, 3 op/s 2024-09-11T08:03:51.792 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[29849]: 22.3 deep-scrub starts 2024-09-11T08:03:51.793 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[29849]: 22.3 deep-scrub ok 2024-09-11T08:03:51.793 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[29849]: from='client.43553 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80700-10", "tierpool":"test-rados-api-smithi102-80700-10-cache"}]': finished 2024-09-11T08:03:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[41371]: 22.6 deep-scrub starts 2024-09-11T08:03:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[41371]: 22.6 deep-scrub ok 2024-09-11T08:03:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[41371]: pgmap v122: 428 pgs: 32 creating+peering, 396 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 18 MiB/s wr, 3 op/s 2024-09-11T08:03:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[41371]: 22.3 deep-scrub starts 2024-09-11T08:03:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[41371]: 22.3 deep-scrub ok 2024-09-11T08:03:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[41371]: from='client.43553 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80700-10", "tierpool":"test-rados-api-smithi102-80700-10-cache"}]': finished 2024-09-11T08:03:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[41371]: from='client.46454 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi102-80338-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:03:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[41371]: osdmap e103: 8 total, 8 up, 8 in 2024-09-11T08:03:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1820385928' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3604557476' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi102-80356-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[41371]: from='client.34895 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:52.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-11"}]: dispatch 2024-09-11T08:03:52.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[41371]: from='client.46460 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi102-80356-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:52.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-11"}]: dispatch 2024-09-11T08:03:52.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:52.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[29849]: from='client.46454 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi102-80338-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:52.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:03:52.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[29849]: osdmap e103: 8 total, 8 up, 8 in 2024-09-11T08:03:52.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1820385928' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:52.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3604557476' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi102-80356-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:52.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[29849]: from='client.34895 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:52.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-11"}]: dispatch 2024-09-11T08:03:52.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[29849]: from='client.46460 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi102-80356-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:52.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-11"}]: dispatch 2024-09-11T08:03:52.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:51 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:52.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:51 smithi146 ceph-mon[33158]: 22.6 deep-scrub starts 2024-09-11T08:03:52.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:51 smithi146 ceph-mon[33158]: 22.6 deep-scrub ok 2024-09-11T08:03:52.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:51 smithi146 ceph-mon[33158]: pgmap v122: 428 pgs: 32 creating+peering, 396 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 18 MiB/s wr, 3 op/s 2024-09-11T08:03:52.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:51 smithi146 ceph-mon[33158]: 22.3 deep-scrub starts 2024-09-11T08:03:52.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:51 smithi146 ceph-mon[33158]: 22.3 deep-scrub ok 2024-09-11T08:03:52.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:51 smithi146 ceph-mon[33158]: from='client.43553 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80700-10", "tierpool":"test-rados-api-smithi102-80700-10-cache"}]': finished 2024-09-11T08:03:52.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:51 smithi146 ceph-mon[33158]: from='client.46454 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi102-80338-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:52.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:51 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:03:52.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:51 smithi146 ceph-mon[33158]: osdmap e103: 8 total, 8 up, 8 in 2024-09-11T08:03:52.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:51 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1820385928' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:52.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:51 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3604557476' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi102-80356-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:52.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:51 smithi146 ceph-mon[33158]: from='client.34895 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:52.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:51 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-11"}]: dispatch 2024-09-11T08:03:52.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:51 smithi146 ceph-mon[33158]: from='client.46460 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi102-80356-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:52.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:51 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-11"}]: dispatch 2024-09-11T08:03:52.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:51 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:52.879 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: Running main() from gmock_main.cc 2024-09-11T08:03:52.879 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-09-11T08:03:52.880 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [----------] Global test environment set-up. 2024-09-11T08:03:52.880 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-09-11T08:03:52.880 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-09-11T08:03:52.880 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [ OK ] LibRadosPools.PoolList (7981 ms) 2024-09-11T08:03:52.880 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-09-11T08:03:52.880 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (2823 ms) 2024-09-11T08:03:52.880 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-09-11T08:03:52.880 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (4239 ms) 2024-09-11T08:03:52.880 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-09-11T08:03:52.880 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (3279 ms) 2024-09-11T08:03:52.880 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-09-11T08:03:52.880 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3278 ms) 2024-09-11T08:03:52.880 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-09-11T08:03:52.880 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (5071 ms) 2024-09-11T08:03:52.881 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-09-11T08:03:52.881 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (5108 ms) 2024-09-11T08:03:52.881 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-09-11T08:03:52.881 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (4853 ms) 2024-09-11T08:03:52.881 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-09-11T08:03:52.881 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (7997 ms) 2024-09-11T08:03:52.881 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [----------] 9 tests from LibRadosPools (44629 ms total) 2024-09-11T08:03:52.881 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: 2024-09-11T08:03:52.881 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [----------] Global test environment tear-down 2024-09-11T08:03:52.881 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (44630 ms total) 2024-09-11T08:03:52.881 INFO:tasks.workunit.client.0.smithi102.stdout: api_pool: [ PASSED ] 9 tests. 2024-09-11T08:03:53.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[41371]: 22.8 deep-scrub starts 2024-09-11T08:03:53.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[41371]: 22.8 deep-scrub ok 2024-09-11T08:03:53.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[41371]: from='client.34895 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]': finished 2024-09-11T08:03:53.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[41371]: from='client.46460 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi102-80356-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:53.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-11"}]': finished 2024-09-11T08:03:53.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[41371]: osdmap e104: 8 total, 8 up, 8 in 2024-09-11T08:03:53.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-11", "mode": "writeback"}]: dispatch 2024-09-11T08:03:53.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-11", "mode": "writeback"}]: dispatch 2024-09-11T08:03:53.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1820385928' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:53.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[41371]: from='client.34895 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:53.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[41371]: 22.4 deep-scrub starts 2024-09-11T08:03:53.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[41371]: 22.4 deep-scrub ok 2024-09-11T08:03:53.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:53.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[29849]: 22.8 deep-scrub starts 2024-09-11T08:03:53.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[29849]: 22.8 deep-scrub ok 2024-09-11T08:03:53.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[29849]: from='client.34895 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]': finished 2024-09-11T08:03:53.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[29849]: from='client.46460 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi102-80356-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:53.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-11"}]': finished 2024-09-11T08:03:53.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[29849]: osdmap e104: 8 total, 8 up, 8 in 2024-09-11T08:03:53.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-11", "mode": "writeback"}]: dispatch 2024-09-11T08:03:53.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-11", "mode": "writeback"}]: dispatch 2024-09-11T08:03:53.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1820385928' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:53.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[29849]: from='client.34895 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:53.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[29849]: 22.4 deep-scrub starts 2024-09-11T08:03:53.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[29849]: 22.4 deep-scrub ok 2024-09-11T08:03:53.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:52 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:53.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:52 smithi146 ceph-mon[33158]: 22.8 deep-scrub starts 2024-09-11T08:03:53.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:52 smithi146 ceph-mon[33158]: 22.8 deep-scrub ok 2024-09-11T08:03:53.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:52 smithi146 ceph-mon[33158]: from='client.34895 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi102-80709-9"}]': finished 2024-09-11T08:03:53.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:52 smithi146 ceph-mon[33158]: from='client.46460 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi102-80356-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:53.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:52 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-11"}]': finished 2024-09-11T08:03:53.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:52 smithi146 ceph-mon[33158]: osdmap e104: 8 total, 8 up, 8 in 2024-09-11T08:03:53.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:52 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-11", "mode": "writeback"}]: dispatch 2024-09-11T08:03:53.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:52 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-11", "mode": "writeback"}]: dispatch 2024-09-11T08:03:53.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:52 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1820385928' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:53.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:52 smithi146 ceph-mon[33158]: from='client.34895 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi102-80709-9"}]: dispatch 2024-09-11T08:03:53.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:52 smithi146 ceph-mon[33158]: 22.4 deep-scrub starts 2024-09-11T08:03:53.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:52 smithi146 ceph-mon[33158]: 22.4 deep-scrub ok 2024-09-11T08:03:53.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:52 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:54.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:53 smithi102 ceph-mon[41371]: pgmap v125: 388 pgs: 32 unknown, 356 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 4 op/s 2024-09-11T08:03:54.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:53 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:03:54.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:53 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-11", "mode": "writeback"}]': finished 2024-09-11T08:03:54.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:53 smithi102 ceph-mon[41371]: from='client.34895 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi102-80709-9"}]': finished 2024-09-11T08:03:54.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:53 smithi102 ceph-mon[41371]: osdmap e105: 8 total, 8 up, 8 in 2024-09-11T08:03:54.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:53 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/263109858' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi102-80338-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:54.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:53 smithi102 ceph-mon[41371]: from='client.46466 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi102-80338-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:54.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:53 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]: dispatch 2024-09-11T08:03:54.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:53 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]: dispatch 2024-09-11T08:03:54.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:53 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi102-80709-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:54.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:53 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:54.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:53 smithi102 ceph-mon[29849]: pgmap v125: 388 pgs: 32 unknown, 356 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 4 op/s 2024-09-11T08:03:54.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:53 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:03:54.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:53 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-11", "mode": "writeback"}]': finished 2024-09-11T08:03:54.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:53 smithi102 ceph-mon[29849]: from='client.34895 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi102-80709-9"}]': finished 2024-09-11T08:03:54.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:53 smithi102 ceph-mon[29849]: osdmap e105: 8 total, 8 up, 8 in 2024-09-11T08:03:54.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:53 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/263109858' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi102-80338-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:54.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:53 smithi102 ceph-mon[29849]: from='client.46466 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi102-80338-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:54.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:53 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]: dispatch 2024-09-11T08:03:54.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:53 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]: dispatch 2024-09-11T08:03:54.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:53 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi102-80709-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:54.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:53 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:54.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:53 smithi146 ceph-mon[33158]: pgmap v125: 388 pgs: 32 unknown, 356 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 4 op/s 2024-09-11T08:03:54.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:53 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:03:54.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:53 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-11", "mode": "writeback"}]': finished 2024-09-11T08:03:54.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:53 smithi146 ceph-mon[33158]: from='client.34895 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi102-80709-9"}]': finished 2024-09-11T08:03:54.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:53 smithi146 ceph-mon[33158]: osdmap e105: 8 total, 8 up, 8 in 2024-09-11T08:03:54.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:53 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/263109858' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi102-80338-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:54.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:53 smithi146 ceph-mon[33158]: from='client.46466 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi102-80338-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:54.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:53 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]: dispatch 2024-09-11T08:03:54.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:53 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]: dispatch 2024-09-11T08:03:54.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:53 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi102-80709-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:03:54.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:53 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:54.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:03:54 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:03:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:03:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:55 smithi102 ceph-mon[29849]: from='client.46466 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi102-80338-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi102-80709-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:55 smithi102 ceph-mon[29849]: osdmap e106: 8 total, 8 up, 8 in 2024-09-11T08:03:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi102-80709-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]: dispatch 2024-09-11T08:03:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2985613920' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi102-80356-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:55 smithi102 ceph-mon[29849]: from='client.47596 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi102-80356-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:03:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:55 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:03:55.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:55.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:55 smithi102 ceph-mon[41371]: from='client.46466 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi102-80338-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:55.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi102-80709-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:55.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:55 smithi102 ceph-mon[41371]: osdmap e106: 8 total, 8 up, 8 in 2024-09-11T08:03:55.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi102-80709-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]: dispatch 2024-09-11T08:03:55.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2985613920' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi102-80356-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:55.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:55 smithi102 ceph-mon[41371]: from='client.47596 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi102-80356-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:55.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:03:55.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:55 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:03:55.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:55 smithi146 ceph-mon[33158]: from='client.46466 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi102-80338-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi102-80709-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:03:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:55 smithi146 ceph-mon[33158]: osdmap e106: 8 total, 8 up, 8 in 2024-09-11T08:03:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi102-80709-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]: dispatch 2024-09-11T08:03:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2985613920' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi102-80356-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:55 smithi146 ceph-mon[33158]: from='client.47596 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi102-80356-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:03:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:55 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:03:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:56.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:56 smithi146 ceph-mon[33158]: pgmap v128: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:03:56.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:56 smithi146 ceph-mon[33158]: from='client.47596 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi102-80356-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:56.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:56 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:03:56.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:56 smithi146 ceph-mon[33158]: osdmap e107: 8 total, 8 up, 8 in 2024-09-11T08:03:56.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:56 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:56.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:56 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-11"}]: dispatch 2024-09-11T08:03:56.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:56 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-11"}]: dispatch 2024-09-11T08:03:56.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:56 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:56.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:56 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:03:56.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:56 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi102-80709-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]': finished 2024-09-11T08:03:56.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:56 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-11"}]': finished 2024-09-11T08:03:56.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:56 smithi146 ceph-mon[33158]: osdmap e108: 8 total, 8 up, 8 in 2024-09-11T08:03:56.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:56 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2647820568' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi102-80338-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:56.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[41371]: pgmap v128: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:03:56.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[41371]: from='client.47596 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi102-80356-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:56.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:03:56.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[41371]: osdmap e107: 8 total, 8 up, 8 in 2024-09-11T08:03:56.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:56.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-11"}]: dispatch 2024-09-11T08:03:56.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-11"}]: dispatch 2024-09-11T08:03:56.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:56.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:03:56.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi102-80709-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]': finished 2024-09-11T08:03:56.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-11"}]': finished 2024-09-11T08:03:56.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[41371]: osdmap e108: 8 total, 8 up, 8 in 2024-09-11T08:03:56.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2647820568' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi102-80338-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:56.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[29849]: pgmap v128: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:03:56.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[29849]: from='client.47596 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi102-80356-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:56.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:03:56.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[29849]: osdmap e107: 8 total, 8 up, 8 in 2024-09-11T08:03:56.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:03:56.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-11"}]: dispatch 2024-09-11T08:03:56.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-11"}]: dispatch 2024-09-11T08:03:56.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:56.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:03:56.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi102-80709-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]': finished 2024-09-11T08:03:56.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-11"}]': finished 2024-09-11T08:03:56.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[29849]: osdmap e108: 8 total, 8 up, 8 in 2024-09-11T08:03:56.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:56 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2647820568' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi102-80338-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:57.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:57 smithi146 ceph-mon[33158]: pgmap v131: 364 pgs: 40 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.0 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-11T08:03:57.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:57 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:57.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:57 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2647820568' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi102-80338-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:57.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:57 smithi146 ceph-mon[33158]: osdmap e109: 8 total, 8 up, 8 in 2024-09-11T08:03:57.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:57 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3997211689' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi102-80356-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:57.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:57 smithi146 ceph-mon[33158]: from='client.46490 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi102-80356-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:57.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:57 smithi102 ceph-mon[41371]: pgmap v131: 364 pgs: 40 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.0 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-11T08:03:57.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:57 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:57.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:57 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2647820568' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi102-80338-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:57.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:57 smithi102 ceph-mon[41371]: osdmap e109: 8 total, 8 up, 8 in 2024-09-11T08:03:57.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:57 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3997211689' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi102-80356-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:57.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:57 smithi102 ceph-mon[41371]: from='client.46490 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi102-80356-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:57.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:57 smithi102 ceph-mon[29849]: pgmap v131: 364 pgs: 40 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.0 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-11T08:03:57.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:57 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:57.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:57 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2647820568' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi102-80338-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:57.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:57 smithi102 ceph-mon[29849]: osdmap e109: 8 total, 8 up, 8 in 2024-09-11T08:03:57.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:57 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3997211689' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi102-80356-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:57.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:57 smithi102 ceph-mon[29849]: from='client.46490 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi102-80356-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:58.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:58 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:58.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:58 smithi146 ceph-mon[33158]: from='client.46490 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi102-80356-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:58.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:58 smithi146 ceph-mon[33158]: osdmap e110: 8 total, 8 up, 8 in 2024-09-11T08:03:58.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:58 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:58.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:58 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:58.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:58 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:58.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:58 smithi102 ceph-mon[41371]: from='client.46490 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi102-80356-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:58.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:58 smithi102 ceph-mon[41371]: osdmap e110: 8 total, 8 up, 8 in 2024-09-11T08:03:58.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:58 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:58.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:58 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:58.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:58 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:58.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:58 smithi102 ceph-mon[29849]: from='client.46490 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi102-80356-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:58.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:58 smithi102 ceph-mon[29849]: osdmap e110: 8 total, 8 up, 8 in 2024-09-11T08:03:58.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:58 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:58.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:58 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:59.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:59 smithi146 ceph-mon[33158]: pgmap v134: 364 pgs: 72 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-11T08:03:59.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:59 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:59.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:59 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:59.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:59 smithi146 ceph-mon[33158]: osdmap e111: 8 total, 8 up, 8 in 2024-09-11T08:03:59.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:59 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1510778313' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi102-80338-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:59.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:03:59 smithi146 ceph-mon[33158]: from='client.47617 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi102-80338-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:59.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:59 smithi102 ceph-mon[29849]: pgmap v134: 364 pgs: 72 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-11T08:03:59.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:59 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:59.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:59 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:59.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:59 smithi102 ceph-mon[29849]: osdmap e111: 8 total, 8 up, 8 in 2024-09-11T08:03:59.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:59 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1510778313' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi102-80338-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:59.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:03:59 smithi102 ceph-mon[29849]: from='client.47617 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi102-80338-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:59.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:59 smithi102 ceph-mon[41371]: pgmap v134: 364 pgs: 72 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-11T08:03:59.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:59 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:03:59.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:59 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:03:59.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:59 smithi102 ceph-mon[41371]: osdmap e111: 8 total, 8 up, 8 in 2024-09-11T08:03:59.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:59 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1510778313' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi102-80338-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:59.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:03:59 smithi102 ceph-mon[41371]: from='client.47617 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi102-80338-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:03:59.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:03:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:04:00.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:03:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:03:59] "GET /metrics HTTP/1.1" 200 240137 "" "Prometheus/2.43.0" 2024-09-11T08:04:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:00 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:04:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:00 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:04:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:00 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:04:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:00 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:00 smithi102 ceph-mon[41371]: from='client.47617 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi102-80338-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:00 smithi102 ceph-mon[41371]: osdmap e112: 8 total, 8 up, 8 in 2024-09-11T08:04:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:00 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2486765775' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi102-80356-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:00 smithi102 ceph-mon[41371]: from='client.47623 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi102-80356-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:00.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:00 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:04:00.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:00 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:04:00.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:00 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:04:00.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:00 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:00.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:00 smithi102 ceph-mon[29849]: from='client.47617 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi102-80338-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:00.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:00 smithi102 ceph-mon[29849]: osdmap e112: 8 total, 8 up, 8 in 2024-09-11T08:04:00.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:00 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2486765775' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi102-80356-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:00.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:00 smithi102 ceph-mon[29849]: from='client.47623 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi102-80356-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:00.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:00 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:04:00.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:00 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:04:00.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:00 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:04:00.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:00 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:00.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:00 smithi146 ceph-mon[33158]: from='client.47617 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi102-80338-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:00.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:00 smithi146 ceph-mon[33158]: osdmap e112: 8 total, 8 up, 8 in 2024-09-11T08:04:00.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:00 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2486765775' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi102-80356-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:00.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:00 smithi146 ceph-mon[33158]: from='client.47623 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi102-80356-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:01.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:01 smithi102 ceph-mon[41371]: pgmap v137: 396 pgs: 32 creating+peering, 364 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 54 KiB/s wr, 50 op/s 2024-09-11T08:04:01.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:01 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:01.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:01 smithi102 ceph-mon[41371]: from='client.47623 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi102-80356-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:01.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:01 smithi102 ceph-mon[41371]: osdmap e113: 8 total, 8 up, 8 in 2024-09-11T08:04:01.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:01 smithi102 ceph-mon[29849]: pgmap v137: 396 pgs: 32 creating+peering, 364 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 54 KiB/s wr, 50 op/s 2024-09-11T08:04:01.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:01 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:01.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:01 smithi102 ceph-mon[29849]: from='client.47623 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi102-80356-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:01.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:01 smithi102 ceph-mon[29849]: osdmap e113: 8 total, 8 up, 8 in 2024-09-11T08:04:01.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:01 smithi146 ceph-mon[33158]: pgmap v137: 396 pgs: 32 creating+peering, 364 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 54 KiB/s wr, 50 op/s 2024-09-11T08:04:01.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:01 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:01.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:01 smithi146 ceph-mon[33158]: from='client.47623 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi102-80356-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:01.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:01 smithi146 ceph-mon[33158]: osdmap e113: 8 total, 8 up, 8 in 2024-09-11T08:04:02.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:02 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:02.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:02 smithi102 ceph-mon[29849]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:02.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:02 smithi102 ceph-mon[29849]: osdmap e114: 8 total, 8 up, 8 in 2024-09-11T08:04:02.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:02 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1296845401' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi102-80338-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:02.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:02 smithi102 ceph-mon[29849]: from='client.47629 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi102-80338-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:02.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:02 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:02.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:02 smithi102 ceph-mon[41371]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:02.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:02 smithi102 ceph-mon[41371]: osdmap e114: 8 total, 8 up, 8 in 2024-09-11T08:04:02.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:02 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1296845401' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi102-80338-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:02.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:02 smithi102 ceph-mon[41371]: from='client.47629 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi102-80338-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:02.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:02 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:02.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:02 smithi146 ceph-mon[33158]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:02.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:02 smithi146 ceph-mon[33158]: osdmap e114: 8 total, 8 up, 8 in 2024-09-11T08:04:02.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:02 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1296845401' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi102-80338-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:02.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:02 smithi146 ceph-mon[33158]: from='client.47629 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi102-80338-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:03.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:03 smithi102 ceph-mon[41371]: pgmap v140: 364 pgs: 32 unknown, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 54 KiB/s wr, 52 op/s 2024-09-11T08:04:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:03 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:03 smithi102 ceph-mon[41371]: from='client.47629 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi102-80338-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:03 smithi102 ceph-mon[41371]: osdmap e115: 8 total, 8 up, 8 in 2024-09-11T08:04:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:03 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/525327387' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi102-80356-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:03 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:04:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:03 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:04:03.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:03 smithi102 ceph-mon[29849]: pgmap v140: 364 pgs: 32 unknown, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 54 KiB/s wr, 52 op/s 2024-09-11T08:04:03.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:03 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:03.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:03 smithi102 ceph-mon[29849]: from='client.47629 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi102-80338-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:03.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:03 smithi102 ceph-mon[29849]: osdmap e115: 8 total, 8 up, 8 in 2024-09-11T08:04:03.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:03 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/525327387' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi102-80356-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:03.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:03 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:04:03.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:03 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:04:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:03 smithi146 ceph-mon[33158]: pgmap v140: 364 pgs: 32 unknown, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 54 KiB/s wr, 52 op/s 2024-09-11T08:04:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:03 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:03 smithi146 ceph-mon[33158]: from='client.47629 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi102-80338-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:03 smithi146 ceph-mon[33158]: osdmap e115: 8 total, 8 up, 8 in 2024-09-11T08:04:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:03 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/525327387' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi102-80356-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:03 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:04:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:03 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:04:04.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:04 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:04.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:04 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/525327387' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi102-80356-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:04.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:04 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:04:04.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:04 smithi102 ceph-mon[41371]: osdmap e116: 8 total, 8 up, 8 in 2024-09-11T08:04:04.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:04 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-13"}]: dispatch 2024-09-11T08:04:04.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:04 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-13"}]: dispatch 2024-09-11T08:04:04.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:04 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:04.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:04 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/525327387' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi102-80356-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:04.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:04 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:04:04.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:04 smithi102 ceph-mon[29849]: osdmap e116: 8 total, 8 up, 8 in 2024-09-11T08:04:04.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:04 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-13"}]: dispatch 2024-09-11T08:04:04.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:04 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-13"}]: dispatch 2024-09-11T08:04:04.684 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:04:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:04:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:04:04.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:04 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:04.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:04 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/525327387' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi102-80356-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:04.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:04 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:04:04.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:04 smithi146 ceph-mon[33158]: osdmap e116: 8 total, 8 up, 8 in 2024-09-11T08:04:04.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:04 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-13"}]: dispatch 2024-09-11T08:04:04.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:04 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-13"}]: dispatch 2024-09-11T08:04:05.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:05 smithi102 ceph-mon[41371]: pgmap v143: 364 pgs: 32 unknown, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:04:05.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:05 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:05.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:05 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-13"}]': finished 2024-09-11T08:04:05.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:05 smithi102 ceph-mon[41371]: osdmap e117: 8 total, 8 up, 8 in 2024-09-11T08:04:05.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:05 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]: dispatch 2024-09-11T08:04:05.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:05 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-13", "mode": "writeback"}]: dispatch 2024-09-11T08:04:05.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:05 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-13", "mode": "writeback"}]: dispatch 2024-09-11T08:04:05.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:05 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1506438559' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi102-80338-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:05.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:05 smithi102 ceph-mon[29849]: pgmap v143: 364 pgs: 32 unknown, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:04:05.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:05 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:05.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:05 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-13"}]': finished 2024-09-11T08:04:05.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:05 smithi102 ceph-mon[29849]: osdmap e117: 8 total, 8 up, 8 in 2024-09-11T08:04:05.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:05 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]: dispatch 2024-09-11T08:04:05.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:05 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-13", "mode": "writeback"}]: dispatch 2024-09-11T08:04:05.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:05 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-13", "mode": "writeback"}]: dispatch 2024-09-11T08:04:05.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:05 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1506438559' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi102-80338-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:05.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:05 smithi146 ceph-mon[33158]: pgmap v143: 364 pgs: 32 unknown, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:04:05.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:05 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:05.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:05 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-13"}]': finished 2024-09-11T08:04:05.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:05 smithi146 ceph-mon[33158]: osdmap e117: 8 total, 8 up, 8 in 2024-09-11T08:04:05.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:05 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]: dispatch 2024-09-11T08:04:05.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:05 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-13", "mode": "writeback"}]: dispatch 2024-09-11T08:04:05.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:05 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-13", "mode": "writeback"}]: dispatch 2024-09-11T08:04:05.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:05 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1506438559' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi102-80338-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:06.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:06 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:06.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:06 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:04:06.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:06 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]': finished 2024-09-11T08:04:06.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:06 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-13", "mode": "writeback"}]': finished 2024-09-11T08:04:06.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:06 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1506438559' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi102-80338-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:06.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:06 smithi102 ceph-mon[29849]: osdmap e118: 8 total, 8 up, 8 in 2024-09-11T08:04:06.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:06 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4282990018' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi102-80356-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:06.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:06 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]: dispatch 2024-09-11T08:04:06.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:06 smithi102 ceph-mon[29849]: from='client.46523 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi102-80356-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:06.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:06 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:06.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:06 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:04:06.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:06 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]': finished 2024-09-11T08:04:06.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:06 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-13", "mode": "writeback"}]': finished 2024-09-11T08:04:06.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:06 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1506438559' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi102-80338-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:06.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:06 smithi102 ceph-mon[41371]: osdmap e118: 8 total, 8 up, 8 in 2024-09-11T08:04:06.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:06 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4282990018' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi102-80356-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:06.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:06 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]: dispatch 2024-09-11T08:04:06.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:06 smithi102 ceph-mon[41371]: from='client.46523 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi102-80356-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:06.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:06 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:06.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:06 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:04:06.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:06 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]': finished 2024-09-11T08:04:06.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:06 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-13", "mode": "writeback"}]': finished 2024-09-11T08:04:06.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:06 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1506438559' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi102-80338-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:06.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:06 smithi146 ceph-mon[33158]: osdmap e118: 8 total, 8 up, 8 in 2024-09-11T08:04:06.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:06 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4282990018' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi102-80356-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:06.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:06 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]: dispatch 2024-09-11T08:04:06.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:06 smithi146 ceph-mon[33158]: from='client.46523 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi102-80356-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:06.905 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: Running main() from gmock_main.cc 2024-09-11T08:04:06.905 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [==========] Running 12 tests from 4 test suites. 2024-09-11T08:04:06.905 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [----------] Global test environment set-up. 2024-09-11T08:04:06.905 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-09-11T08:04:06.905 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-09-11T08:04:06.905 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (3076 ms) 2024-09-11T08:04:06.905 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-09-11T08:04:06.905 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (2973 ms) 2024-09-11T08:04:06.905 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-09-11T08:04:06.905 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (2285 ms) 2024-09-11T08:04:06.905 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-09-11T08:04:06.905 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (2019 ms) 2024-09-11T08:04:06.906 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (10353 ms total) 2024-09-11T08:04:06.906 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: 2024-09-11T08:04:06.906 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManaged 2024-09-11T08:04:06.906 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-09-11T08:04:06.906 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (4088 ms) 2024-09-11T08:04:06.906 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-09-11T08:04:06.906 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (4015 ms) 2024-09-11T08:04:06.906 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManaged (8103 ms total) 2024-09-11T08:04:06.906 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: 2024-09-11T08:04:06.906 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-09-11T08:04:06.906 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-09-11T08:04:06.906 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (2770 ms) 2024-09-11T08:04:06.906 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-09-11T08:04:06.906 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (2005 ms) 2024-09-11T08:04:06.907 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-09-11T08:04:06.907 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (2007 ms) 2024-09-11T08:04:06.907 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-09-11T08:04:06.907 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (2008 ms) 2024-09-11T08:04:06.907 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (8790 ms total) 2024-09-11T08:04:06.907 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: 2024-09-11T08:04:06.907 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-09-11T08:04:06.907 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-09-11T08:04:06.907 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (4026 ms) 2024-09-11T08:04:06.907 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-09-11T08:04:06.907 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (3993 ms) 2024-09-11T08:04:06.907 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8019 ms total) 2024-09-11T08:04:06.907 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: 2024-09-11T08:04:06.907 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [----------] Global test environment tear-down 2024-09-11T08:04:06.908 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [==========] 12 tests from 4 test suites ran. (58659 ms total) 2024-09-11T08:04:06.908 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots: [ PASSED ] 12 tests. 2024-09-11T08:04:07.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: pgmap v146: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 71 KiB/s wr, 75 op/s 2024-09-11T08:04:07.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.0"}]: dispatch 2024-09-11T08:04:07.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.0"}]: dispatch 2024-09-11T08:04:07.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.1"}]: dispatch 2024-09-11T08:04:07.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.1"}]: dispatch 2024-09-11T08:04:07.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.2"}]: dispatch 2024-09-11T08:04:07.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.2"}]: dispatch 2024-09-11T08:04:07.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.3"}]: dispatch 2024-09-11T08:04:07.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.3"}]: dispatch 2024-09-11T08:04:07.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.4"}]: dispatch 2024-09-11T08:04:07.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.4"}]: dispatch 2024-09-11T08:04:07.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.5"}]: dispatch 2024-09-11T08:04:07.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.5"}]: dispatch 2024-09-11T08:04:07.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.6"}]: dispatch 2024-09-11T08:04:07.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.6"}]: dispatch 2024-09-11T08:04:07.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.7"}]: dispatch 2024-09-11T08:04:07.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.7"}]: dispatch 2024-09-11T08:04:07.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.8"}]: dispatch 2024-09-11T08:04:07.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.8"}]: dispatch 2024-09-11T08:04:07.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.9"}]: dispatch 2024-09-11T08:04:07.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.9"}]: dispatch 2024-09-11T08:04:07.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:07.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:07.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]': finished 2024-09-11T08:04:07.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: from='client.46523 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi102-80356-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:07.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[41371]: osdmap e119: 8 total, 8 up, 8 in 2024-09-11T08:04:07.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: pgmap v146: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 71 KiB/s wr, 75 op/s 2024-09-11T08:04:07.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.0"}]: dispatch 2024-09-11T08:04:07.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.0"}]: dispatch 2024-09-11T08:04:07.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.1"}]: dispatch 2024-09-11T08:04:07.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.1"}]: dispatch 2024-09-11T08:04:07.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.2"}]: dispatch 2024-09-11T08:04:07.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.2"}]: dispatch 2024-09-11T08:04:07.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.3"}]: dispatch 2024-09-11T08:04:07.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.3"}]: dispatch 2024-09-11T08:04:07.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.4"}]: dispatch 2024-09-11T08:04:07.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.4"}]: dispatch 2024-09-11T08:04:07.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.5"}]: dispatch 2024-09-11T08:04:07.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.5"}]: dispatch 2024-09-11T08:04:07.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.6"}]: dispatch 2024-09-11T08:04:07.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.6"}]: dispatch 2024-09-11T08:04:07.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.7"}]: dispatch 2024-09-11T08:04:07.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.7"}]: dispatch 2024-09-11T08:04:07.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.8"}]: dispatch 2024-09-11T08:04:07.651 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.8"}]: dispatch 2024-09-11T08:04:07.651 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.9"}]: dispatch 2024-09-11T08:04:07.651 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.9"}]: dispatch 2024-09-11T08:04:07.651 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:07.651 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:07.651 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]': finished 2024-09-11T08:04:07.651 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: from='client.46523 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi102-80356-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:07.651 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:07 smithi102 ceph-mon[29849]: osdmap e119: 8 total, 8 up, 8 in 2024-09-11T08:04:07.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: pgmap v146: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 71 KiB/s wr, 75 op/s 2024-09-11T08:04:07.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.0"}]: dispatch 2024-09-11T08:04:07.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.0"}]: dispatch 2024-09-11T08:04:07.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.1"}]: dispatch 2024-09-11T08:04:07.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.1"}]: dispatch 2024-09-11T08:04:07.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.2"}]: dispatch 2024-09-11T08:04:07.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.2"}]: dispatch 2024-09-11T08:04:07.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.3"}]: dispatch 2024-09-11T08:04:07.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.3"}]: dispatch 2024-09-11T08:04:07.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.4"}]: dispatch 2024-09-11T08:04:07.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.4"}]: dispatch 2024-09-11T08:04:07.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.5"}]: dispatch 2024-09-11T08:04:07.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.5"}]: dispatch 2024-09-11T08:04:07.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.6"}]: dispatch 2024-09-11T08:04:07.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.6"}]: dispatch 2024-09-11T08:04:07.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.7"}]: dispatch 2024-09-11T08:04:07.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.7"}]: dispatch 2024-09-11T08:04:07.686 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.8"}]: dispatch 2024-09-11T08:04:07.686 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.8"}]: dispatch 2024-09-11T08:04:07.686 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "98.9"}]: dispatch 2024-09-11T08:04:07.686 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "98.9"}]: dispatch 2024-09-11T08:04:07.686 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:07.686 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:07.686 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1137315399' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi102-80709-14"}]': finished 2024-09-11T08:04:07.686 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: from='client.46523 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi102-80356-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:07.686 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:07 smithi146 ceph-mon[33158]: osdmap e119: 8 total, 8 up, 8 in 2024-09-11T08:04:08.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:08 smithi102 ceph-mon[41371]: 98.4 scrub starts 2024-09-11T08:04:08.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:08 smithi102 ceph-mon[41371]: 98.4 scrub ok 2024-09-11T08:04:08.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:08 smithi102 ceph-mon[41371]: 98.0 scrub starts 2024-09-11T08:04:08.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:08 smithi102 ceph-mon[41371]: 98.0 scrub ok 2024-09-11T08:04:08.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:08 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:08.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:08 smithi102 ceph-mon[41371]: osdmap e120: 8 total, 8 up, 8 in 2024-09-11T08:04:08.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:08 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2936773028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi102-80338-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:08.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:08 smithi102 ceph-mon[41371]: from='client.46529 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi102-80338-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:08.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:08 smithi102 ceph-mon[29849]: 98.4 scrub starts 2024-09-11T08:04:08.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:08 smithi102 ceph-mon[29849]: 98.4 scrub ok 2024-09-11T08:04:08.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:08 smithi102 ceph-mon[29849]: 98.0 scrub starts 2024-09-11T08:04:08.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:08 smithi102 ceph-mon[29849]: 98.0 scrub ok 2024-09-11T08:04:08.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:08 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:08.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:08 smithi102 ceph-mon[29849]: osdmap e120: 8 total, 8 up, 8 in 2024-09-11T08:04:08.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:08 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2936773028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi102-80338-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:08.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:08 smithi102 ceph-mon[29849]: from='client.46529 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi102-80338-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:08.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:08 smithi146 ceph-mon[33158]: 98.4 scrub starts 2024-09-11T08:04:08.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:08 smithi146 ceph-mon[33158]: 98.4 scrub ok 2024-09-11T08:04:08.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:08 smithi146 ceph-mon[33158]: 98.0 scrub starts 2024-09-11T08:04:08.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:08 smithi146 ceph-mon[33158]: 98.0 scrub ok 2024-09-11T08:04:08.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:08 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:08.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:08 smithi146 ceph-mon[33158]: osdmap e120: 8 total, 8 up, 8 in 2024-09-11T08:04:08.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:08 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2936773028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi102-80338-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:08.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:08 smithi146 ceph-mon[33158]: from='client.46529 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi102-80338-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:09.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:09 smithi102 ceph-mon[41371]: 98.5 scrub starts 2024-09-11T08:04:09.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:09 smithi102 ceph-mon[41371]: 98.5 scrub ok 2024-09-11T08:04:09.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:09 smithi102 ceph-mon[41371]: pgmap v149: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 71 KiB/s wr, 75 op/s 2024-09-11T08:04:09.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:09 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:09.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:09 smithi102 ceph-mon[41371]: from='client.46529 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi102-80338-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:09.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:09 smithi102 ceph-mon[41371]: osdmap e121: 8 total, 8 up, 8 in 2024-09-11T08:04:09.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:09 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3117919490' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi102-80356-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:09.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:09 smithi102 ceph-mon[41371]: from='client.46535 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi102-80356-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:09.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:09 smithi102 ceph-mon[29849]: 98.5 scrub starts 2024-09-11T08:04:09.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:09 smithi102 ceph-mon[29849]: 98.5 scrub ok 2024-09-11T08:04:09.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:09 smithi102 ceph-mon[29849]: pgmap v149: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 71 KiB/s wr, 75 op/s 2024-09-11T08:04:09.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:09 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:09.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:09 smithi102 ceph-mon[29849]: from='client.46529 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi102-80338-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:09.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:09 smithi102 ceph-mon[29849]: osdmap e121: 8 total, 8 up, 8 in 2024-09-11T08:04:09.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:09 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3117919490' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi102-80356-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:09.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:09 smithi102 ceph-mon[29849]: from='client.46535 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi102-80356-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:09.684 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:04:09 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:04:09.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:09 smithi146 ceph-mon[33158]: 98.5 scrub starts 2024-09-11T08:04:09.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:09 smithi146 ceph-mon[33158]: 98.5 scrub ok 2024-09-11T08:04:09.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:09 smithi146 ceph-mon[33158]: pgmap v149: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 71 KiB/s wr, 75 op/s 2024-09-11T08:04:09.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:09 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:09.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:09 smithi146 ceph-mon[33158]: from='client.46529 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi102-80338-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:09.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:09 smithi146 ceph-mon[33158]: osdmap e121: 8 total, 8 up, 8 in 2024-09-11T08:04:09.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:09 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3117919490' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi102-80356-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:09.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:09 smithi146 ceph-mon[33158]: from='client.46535 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi102-80356-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:10.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:04:09 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:04:09] "GET /metrics HTTP/1.1" 200 240137 "" "Prometheus/2.43.0" 2024-09-11T08:04:10.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:10 smithi102 ceph-mon[29849]: 98.2 scrub starts 2024-09-11T08:04:10.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:10 smithi102 ceph-mon[29849]: 98.2 scrub ok 2024-09-11T08:04:10.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:10 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:04:10.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:10.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:10 smithi102 ceph-mon[29849]: from='client.46535 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi102-80356-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:10.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:10 smithi102 ceph-mon[29849]: osdmap e122: 8 total, 8 up, 8 in 2024-09-11T08:04:10.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:10 smithi102 ceph-mon[41371]: 98.2 scrub starts 2024-09-11T08:04:10.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:10 smithi102 ceph-mon[41371]: 98.2 scrub ok 2024-09-11T08:04:10.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:10 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:04:10.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:10.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:10 smithi102 ceph-mon[41371]: from='client.46535 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi102-80356-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:10.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:10 smithi102 ceph-mon[41371]: osdmap e122: 8 total, 8 up, 8 in 2024-09-11T08:04:10.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:10 smithi146 ceph-mon[33158]: 98.2 scrub starts 2024-09-11T08:04:10.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:10 smithi146 ceph-mon[33158]: 98.2 scrub ok 2024-09-11T08:04:10.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:10 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:04:10.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:10.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:10 smithi146 ceph-mon[33158]: from='client.46535 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi102-80356-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:10.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:10 smithi146 ceph-mon[33158]: osdmap e122: 8 total, 8 up, 8 in 2024-09-11T08:04:11.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:11 smithi102 ceph-mon[29849]: 98.1 scrub starts 2024-09-11T08:04:11.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:11 smithi102 ceph-mon[29849]: 98.1 scrub ok 2024-09-11T08:04:11.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:11 smithi102 ceph-mon[29849]: pgmap v152: 356 pgs: 356 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 57 KiB/s rd, 255 B/s wr, 77 op/s 2024-09-11T08:04:11.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:11 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:11.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:11 smithi102 ceph-mon[29849]: osdmap e123: 8 total, 8 up, 8 in 2024-09-11T08:04:11.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:11 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3131363073' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi102-80338-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:11.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:11 smithi102 ceph-mon[29849]: from='client.47665 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi102-80338-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:11.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:11 smithi102 ceph-mon[41371]: 98.1 scrub starts 2024-09-11T08:04:11.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:11 smithi102 ceph-mon[41371]: 98.1 scrub ok 2024-09-11T08:04:11.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:11 smithi102 ceph-mon[41371]: pgmap v152: 356 pgs: 356 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 57 KiB/s rd, 255 B/s wr, 77 op/s 2024-09-11T08:04:11.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:11 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:11.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:11 smithi102 ceph-mon[41371]: osdmap e123: 8 total, 8 up, 8 in 2024-09-11T08:04:11.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:11 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3131363073' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi102-80338-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:11.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:11 smithi102 ceph-mon[41371]: from='client.47665 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi102-80338-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:11.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:11 smithi146 ceph-mon[33158]: 98.1 scrub starts 2024-09-11T08:04:11.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:11 smithi146 ceph-mon[33158]: 98.1 scrub ok 2024-09-11T08:04:11.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:11 smithi146 ceph-mon[33158]: pgmap v152: 356 pgs: 356 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 57 KiB/s rd, 255 B/s wr, 77 op/s 2024-09-11T08:04:11.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:11 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:11.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:11 smithi146 ceph-mon[33158]: osdmap e123: 8 total, 8 up, 8 in 2024-09-11T08:04:11.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:11 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3131363073' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi102-80338-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:11.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:11 smithi146 ceph-mon[33158]: from='client.47665 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi102-80338-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:12.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[29849]: 98.6 scrub starts 2024-09-11T08:04:12.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[29849]: 98.6 scrub ok 2024-09-11T08:04:12.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[29849]: 98.8 scrub starts 2024-09-11T08:04:12.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[29849]: 98.8 scrub ok 2024-09-11T08:04:12.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[29849]: 98.7 scrub starts 2024-09-11T08:04:12.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[29849]: 98.7 scrub ok 2024-09-11T08:04:12.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[29849]: 98.3 scrub starts 2024-09-11T08:04:12.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[29849]: 98.3 scrub ok 2024-09-11T08:04:12.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:12.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:04:12.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[29849]: from='client.24692 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:04:12.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[29849]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:12.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[29849]: from='client.47665 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi102-80338-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:12.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[29849]: from='client.24692 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pg_num","val":"11"}]': finished 2024-09-11T08:04:12.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[29849]: osdmap e124: 8 total, 8 up, 8 in 2024-09-11T08:04:12.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4101730016' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi102-80356-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:12.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:12.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:12.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1354076032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80697-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:12.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[29849]: from='client.47668 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi102-80356-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:12.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[29849]: from='client.47674 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80697-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:12.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:04:12.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[29849]: from='client.24692 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:04:12.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[41371]: 98.6 scrub starts 2024-09-11T08:04:12.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[41371]: 98.6 scrub ok 2024-09-11T08:04:12.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[41371]: 98.8 scrub starts 2024-09-11T08:04:12.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[41371]: 98.8 scrub ok 2024-09-11T08:04:12.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[41371]: 98.7 scrub starts 2024-09-11T08:04:12.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[41371]: 98.7 scrub ok 2024-09-11T08:04:12.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[41371]: 98.3 scrub starts 2024-09-11T08:04:12.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[41371]: 98.3 scrub ok 2024-09-11T08:04:12.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:12.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:04:12.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[41371]: from='client.24692 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:04:12.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[41371]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:12.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[41371]: from='client.47665 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi102-80338-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:12.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[41371]: from='client.24692 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pg_num","val":"11"}]': finished 2024-09-11T08:04:12.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[41371]: osdmap e124: 8 total, 8 up, 8 in 2024-09-11T08:04:12.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4101730016' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi102-80356-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:12.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:12.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:12.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1354076032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80697-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:12.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[41371]: from='client.47668 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi102-80356-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:12.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[41371]: from='client.47674 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80697-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:12.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:04:12.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:12 smithi102 ceph-mon[41371]: from='client.24692 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:04:12.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:12 smithi146 ceph-mon[33158]: 98.6 scrub starts 2024-09-11T08:04:12.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:12 smithi146 ceph-mon[33158]: 98.6 scrub ok 2024-09-11T08:04:12.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:12 smithi146 ceph-mon[33158]: 98.8 scrub starts 2024-09-11T08:04:12.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:12 smithi146 ceph-mon[33158]: 98.8 scrub ok 2024-09-11T08:04:12.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:12 smithi146 ceph-mon[33158]: 98.7 scrub starts 2024-09-11T08:04:12.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:12 smithi146 ceph-mon[33158]: 98.7 scrub ok 2024-09-11T08:04:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:12 smithi146 ceph-mon[33158]: 98.3 scrub starts 2024-09-11T08:04:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:12 smithi146 ceph-mon[33158]: 98.3 scrub ok 2024-09-11T08:04:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:04:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:12 smithi146 ceph-mon[33158]: from='client.24692 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:04:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:12 smithi146 ceph-mon[33158]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:12 smithi146 ceph-mon[33158]: from='client.47665 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi102-80338-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:12 smithi146 ceph-mon[33158]: from='client.24692 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pg_num","val":"11"}]': finished 2024-09-11T08:04:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:12 smithi146 ceph-mon[33158]: osdmap e124: 8 total, 8 up, 8 in 2024-09-11T08:04:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4101730016' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi102-80356-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:12.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1354076032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80697-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:12.936 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:12 smithi146 ceph-mon[33158]: from='client.47668 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi102-80356-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:12.936 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:12 smithi146 ceph-mon[33158]: from='client.47674 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80697-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:12.936 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:04:12.936 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:12 smithi146 ceph-mon[33158]: from='client.24692 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:04:13.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:13 smithi102 ceph-mon[29849]: pgmap v155: 388 pgs: 96 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 56 KiB/s rd, 73 op/s 2024-09-11T08:04:13.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:13 smithi102 ceph-mon[29849]: from='client.47668 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi102-80356-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:13.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:13 smithi102 ceph-mon[29849]: from='client.47674 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80697-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:13.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:13 smithi102 ceph-mon[29849]: from='client.24692 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pgp_num","val":"11"}]': finished 2024-09-11T08:04:13.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:13 smithi102 ceph-mon[29849]: osdmap e125: 8 total, 8 up, 8 in 2024-09-11T08:04:13.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:13.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:13 smithi102 ceph-mon[41371]: pgmap v155: 388 pgs: 96 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 56 KiB/s rd, 73 op/s 2024-09-11T08:04:13.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:13 smithi102 ceph-mon[41371]: from='client.47668 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi102-80356-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:13.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:13 smithi102 ceph-mon[41371]: from='client.47674 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80697-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:13.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:13 smithi102 ceph-mon[41371]: from='client.24692 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pgp_num","val":"11"}]': finished 2024-09-11T08:04:13.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:13 smithi102 ceph-mon[41371]: osdmap e125: 8 total, 8 up, 8 in 2024-09-11T08:04:13.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:13.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:13 smithi146 ceph-mon[33158]: pgmap v155: 388 pgs: 96 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 56 KiB/s rd, 73 op/s 2024-09-11T08:04:13.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:13 smithi146 ceph-mon[33158]: from='client.47668 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi102-80356-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:13.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:13 smithi146 ceph-mon[33158]: from='client.47674 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80697-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:13.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:13 smithi146 ceph-mon[33158]: from='client.24692 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pgp_num","val":"11"}]': finished 2024-09-11T08:04:13.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:13 smithi146 ceph-mon[33158]: osdmap e125: 8 total, 8 up, 8 in 2024-09-11T08:04:13.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:14.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:04:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:04:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:04:15.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:14 smithi102 ceph-mon[41371]: osdmap e126: 8 total, 8 up, 8 in 2024-09-11T08:04:15.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:15.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1302072452' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi102-80338-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:15.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:14 smithi102 ceph-mon[41371]: from='client.47683 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi102-80338-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:15.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:14 smithi102 ceph-mon[41371]: pgmap v158: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:04:15.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:14 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:04:15.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:14 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:04:15.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:14 smithi102 ceph-mon[29849]: osdmap e126: 8 total, 8 up, 8 in 2024-09-11T08:04:15.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:15.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1302072452' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi102-80338-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:15.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:14 smithi102 ceph-mon[29849]: from='client.47683 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi102-80338-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:15.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:14 smithi102 ceph-mon[29849]: pgmap v158: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:04:15.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:14 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:04:15.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:14 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:04:15.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:14 smithi146 ceph-mon[33158]: osdmap e126: 8 total, 8 up, 8 in 2024-09-11T08:04:15.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:15.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1302072452' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi102-80338-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:15.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:14 smithi146 ceph-mon[33158]: from='client.47683 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi102-80338-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:15.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:14 smithi146 ceph-mon[33158]: pgmap v158: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:04:15.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:14 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:04:15.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:14 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:04:16.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:15 smithi102 ceph-mon[29849]: from='client.47683 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi102-80338-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:16.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:15 smithi102 ceph-mon[29849]: osdmap e127: 8 total, 8 up, 8 in 2024-09-11T08:04:16.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:15 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:16.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:15 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/258320585' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi102-80356-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:16.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:15 smithi102 ceph-mon[29849]: from='client.46571 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi102-80356-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:16.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:15 smithi102 ceph-mon[29849]: 98.9 scrub starts 2024-09-11T08:04:16.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:15 smithi102 ceph-mon[29849]: 98.9 scrub ok 2024-09-11T08:04:16.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:15 smithi102 ceph-mon[41371]: from='client.47683 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi102-80338-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:16.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:15 smithi102 ceph-mon[41371]: osdmap e127: 8 total, 8 up, 8 in 2024-09-11T08:04:16.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:15 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:16.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:15 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/258320585' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi102-80356-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:16.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:15 smithi102 ceph-mon[41371]: from='client.46571 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi102-80356-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:16.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:15 smithi102 ceph-mon[41371]: 98.9 scrub starts 2024-09-11T08:04:16.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:15 smithi102 ceph-mon[41371]: 98.9 scrub ok 2024-09-11T08:04:16.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:15 smithi146 ceph-mon[33158]: from='client.47683 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi102-80338-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:16.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:15 smithi146 ceph-mon[33158]: osdmap e127: 8 total, 8 up, 8 in 2024-09-11T08:04:16.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:15 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:16.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:15 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/258320585' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi102-80356-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:16.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:15 smithi146 ceph-mon[33158]: from='client.46571 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi102-80356-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:16.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:15 smithi146 ceph-mon[33158]: 98.9 scrub starts 2024-09-11T08:04:16.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:15 smithi146 ceph-mon[33158]: 98.9 scrub ok 2024-09-11T08:04:17.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:17.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[29849]: from='client.46571 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi102-80356-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:17.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[29849]: osdmap e128: 8 total, 8 up, 8 in 2024-09-11T08:04:17.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1275883381' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:17.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[29849]: from='client.46595 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:17.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1275883381' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:17.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[29849]: from='client.46595 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:17.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1275883381' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:17.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[29849]: from='client.46595 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:17.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[29849]: pgmap v161: 324 pgs: 32 creating+peering, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:04:17.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:17.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:17.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:17.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[41371]: from='client.46571 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi102-80356-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:17.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[41371]: osdmap e128: 8 total, 8 up, 8 in 2024-09-11T08:04:17.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1275883381' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:17.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[41371]: from='client.46595 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:17.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1275883381' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:17.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[41371]: from='client.46595 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:17.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1275883381' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:17.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[41371]: from='client.46595 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:17.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[41371]: pgmap v161: 324 pgs: 32 creating+peering, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:04:17.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:17.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:17.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:17.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:16 smithi146 ceph-mon[33158]: from='client.46571 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi102-80356-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:17.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:16 smithi146 ceph-mon[33158]: osdmap e128: 8 total, 8 up, 8 in 2024-09-11T08:04:17.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1275883381' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:17.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:16 smithi146 ceph-mon[33158]: from='client.46595 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:17.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1275883381' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:17.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:16 smithi146 ceph-mon[33158]: from='client.46595 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:17.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1275883381' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:17.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:16 smithi146 ceph-mon[33158]: from='client.46595 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:17.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:16 smithi146 ceph-mon[33158]: pgmap v161: 324 pgs: 32 creating+peering, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:04:17.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:16 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:17.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:18.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:18 smithi102 ceph-mon[41371]: from='client.46595 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:18.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:18 smithi102 ceph-mon[41371]: osdmap e129: 8 total, 8 up, 8 in 2024-09-11T08:04:18.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1275883381' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi102-80697-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:18.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:18 smithi102 ceph-mon[41371]: from='client.46595 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi102-80697-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:18.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2386473643' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi102-80338-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:18.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:18 smithi102 ceph-mon[41371]: from='client.46586 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi102-80338-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:18.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T08:04:18.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:18.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:18 smithi102 ceph-mon[41371]: from='client.24745 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:18.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:18.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:18 smithi102 ceph-mon[29849]: from='client.46595 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:18.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:18 smithi102 ceph-mon[29849]: osdmap e129: 8 total, 8 up, 8 in 2024-09-11T08:04:18.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1275883381' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi102-80697-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:18.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:18 smithi102 ceph-mon[29849]: from='client.46595 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi102-80697-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:18.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2386473643' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi102-80338-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:18.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:18 smithi102 ceph-mon[29849]: from='client.46586 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi102-80338-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:18.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T08:04:18.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:18.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:18 smithi102 ceph-mon[29849]: from='client.24745 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:18.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:18.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:18 smithi146 ceph-mon[33158]: from='client.46595 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:18.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:18 smithi146 ceph-mon[33158]: osdmap e129: 8 total, 8 up, 8 in 2024-09-11T08:04:18.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1275883381' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi102-80697-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:18.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:18 smithi146 ceph-mon[33158]: from='client.46595 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi102-80697-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:18.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2386473643' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi102-80338-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:18.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:18 smithi146 ceph-mon[33158]: from='client.46586 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi102-80338-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:18.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T08:04:18.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:18.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:18 smithi146 ceph-mon[33158]: from='client.24745 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:18.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:19.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[41371]: from='client.46586 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi102-80338-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:19.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[41371]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:19.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[41371]: osdmap e130: 8 total, 8 up, 8 in 2024-09-11T08:04:19.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2851535643' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi102-80356-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:19.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "app2"}]: dispatch 2024-09-11T08:04:19.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:19.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[41371]: from='client.46601 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi102-80356-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:19.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[41371]: from='client.24745 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:19.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[41371]: pgmap v164: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:04:19.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:19.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[41371]: from='client.46595 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi102-80697-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]': finished 2024-09-11T08:04:19.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[41371]: from='client.46601 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi102-80356-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:19.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[41371]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:19.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[41371]: osdmap e131: 8 total, 8 up, 8 in 2024-09-11T08:04:19.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi102-80691-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:04:19.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:04:19.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[41371]: from='client.24745 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:04:19.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[29849]: from='client.46586 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi102-80338-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:19.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[29849]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:19.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[29849]: osdmap e130: 8 total, 8 up, 8 in 2024-09-11T08:04:19.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2851535643' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi102-80356-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:19.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "app2"}]: dispatch 2024-09-11T08:04:19.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:19.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[29849]: from='client.46601 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi102-80356-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:19.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[29849]: from='client.24745 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:19.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[29849]: pgmap v164: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:04:19.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:19.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[29849]: from='client.46595 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi102-80697-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]': finished 2024-09-11T08:04:19.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[29849]: from='client.46601 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi102-80356-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:19.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[29849]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:19.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[29849]: osdmap e131: 8 total, 8 up, 8 in 2024-09-11T08:04:19.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi102-80691-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:04:19.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:04:19.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-mon[29849]: from='client.24745 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:04:19.684 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:04:19 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:04:19.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:19 smithi146 ceph-mon[33158]: from='client.46586 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi102-80338-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:19.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:19 smithi146 ceph-mon[33158]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:19.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:19 smithi146 ceph-mon[33158]: osdmap e130: 8 total, 8 up, 8 in 2024-09-11T08:04:19.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2851535643' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi102-80356-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:19.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "app2"}]: dispatch 2024-09-11T08:04:19.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:19.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:19 smithi146 ceph-mon[33158]: from='client.46601 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi102-80356-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:19.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:19 smithi146 ceph-mon[33158]: from='client.24745 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:19.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:19 smithi146 ceph-mon[33158]: pgmap v164: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:04:19.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:19.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:19 smithi146 ceph-mon[33158]: from='client.46595 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi102-80697-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]': finished 2024-09-11T08:04:19.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:19 smithi146 ceph-mon[33158]: from='client.46601 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi102-80356-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:19.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:19 smithi146 ceph-mon[33158]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi102-80691-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:19.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:19 smithi146 ceph-mon[33158]: osdmap e131: 8 total, 8 up, 8 in 2024-09-11T08:04:19.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi102-80691-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:04:19.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:04:19.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:19 smithi146 ceph-mon[33158]: from='client.24745 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-11T08:04:20.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:04:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:04:19] "GET /metrics HTTP/1.1" 200 237987 "" "Prometheus/2.43.0" 2024-09-11T08:04:20.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:20 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:04:20.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:20.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:20 smithi102 ceph-mon[29849]: from='client.24745 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-11T08:04:20.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:20 smithi102 ceph-mon[29849]: osdmap e132: 8 total, 8 up, 8 in 2024-09-11T08:04:20.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3556581938' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi102-80338-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:20.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-11T08:04:20.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:20 smithi102 ceph-mon[29849]: from='client.24745 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-11T08:04:20.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:20 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:04:20.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:20.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:20 smithi102 ceph-mon[41371]: from='client.24745 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-11T08:04:20.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:20 smithi102 ceph-mon[41371]: osdmap e132: 8 total, 8 up, 8 in 2024-09-11T08:04:20.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3556581938' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi102-80338-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:20.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-11T08:04:20.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:20 smithi102 ceph-mon[41371]: from='client.24745 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-11T08:04:20.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:20 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:04:20.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:20.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:20 smithi146 ceph-mon[33158]: from='client.24745 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-11T08:04:20.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:20 smithi146 ceph-mon[33158]: osdmap e132: 8 total, 8 up, 8 in 2024-09-11T08:04:20.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3556581938' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi102-80338-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:20.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-11T08:04:20.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:20 smithi146 ceph-mon[33158]: from='client.24745 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-11T08:04:21.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:21 smithi102 ceph-mon[41371]: pgmap v167: 332 pgs: 32 creating+peering, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:04:21.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:21 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:21.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:21 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3556581938' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi102-80338-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:21.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:21 smithi102 ceph-mon[41371]: from='client.24745 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-11T08:04:21.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:21 smithi102 ceph-mon[41371]: osdmap e133: 8 total, 8 up, 8 in 2024-09-11T08:04:21.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:21 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key1"}]: dispatch 2024-09-11T08:04:21.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:21 smithi102 ceph-mon[41371]: from='client.24745 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key1"}]: dispatch 2024-09-11T08:04:21.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:21 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1116584906' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi102-80356-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:21.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:21 smithi102 ceph-mon[41371]: from='client.46613 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi102-80356-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:21.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:21 smithi102 ceph-mon[29849]: pgmap v167: 332 pgs: 32 creating+peering, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:04:21.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:21 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:21.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:21 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3556581938' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi102-80338-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:21.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:21 smithi102 ceph-mon[29849]: from='client.24745 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-11T08:04:21.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:21 smithi102 ceph-mon[29849]: osdmap e133: 8 total, 8 up, 8 in 2024-09-11T08:04:21.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:21 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key1"}]: dispatch 2024-09-11T08:04:21.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:21 smithi102 ceph-mon[29849]: from='client.24745 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key1"}]: dispatch 2024-09-11T08:04:21.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:21 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1116584906' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi102-80356-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:21.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:21 smithi102 ceph-mon[29849]: from='client.46613 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi102-80356-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:21.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:21 smithi146 ceph-mon[33158]: pgmap v167: 332 pgs: 32 creating+peering, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:04:21.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:21 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:21.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:21 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3556581938' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi102-80338-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:21.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:21 smithi146 ceph-mon[33158]: from='client.24745 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-11T08:04:21.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:21 smithi146 ceph-mon[33158]: osdmap e133: 8 total, 8 up, 8 in 2024-09-11T08:04:21.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:21 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3646719286' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key1"}]: dispatch 2024-09-11T08:04:21.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:21 smithi146 ceph-mon[33158]: from='client.24745 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key1"}]: dispatch 2024-09-11T08:04:21.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:21 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1116584906' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi102-80356-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:21.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:21 smithi146 ceph-mon[33158]: from='client.46613 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi102-80356-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:22.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:22 smithi102 ceph-mon[41371]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:22.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:22.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:22 smithi102 ceph-mon[41371]: from='client.24745 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key1"}]': finished 2024-09-11T08:04:22.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:22 smithi102 ceph-mon[41371]: from='client.46613 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi102-80356-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:22.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:22 smithi102 ceph-mon[41371]: osdmap e134: 8 total, 8 up, 8 in 2024-09-11T08:04:22.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:22 smithi102 ceph-mon[29849]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:22.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:22.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:22 smithi102 ceph-mon[29849]: from='client.24745 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key1"}]': finished 2024-09-11T08:04:22.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:22 smithi102 ceph-mon[29849]: from='client.46613 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi102-80356-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:22.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:22 smithi102 ceph-mon[29849]: osdmap e134: 8 total, 8 up, 8 in 2024-09-11T08:04:22.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:22 smithi146 ceph-mon[33158]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:22.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:22.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:22 smithi146 ceph-mon[33158]: from='client.24745 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi102-80691-1","app":"app1","key":"key1"}]': finished 2024-09-11T08:04:22.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:22 smithi146 ceph-mon[33158]: from='client.46613 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi102-80356-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:22.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:22 smithi146 ceph-mon[33158]: osdmap e134: 8 total, 8 up, 8 in 2024-09-11T08:04:22.980 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-09-11T08:04:22.981 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-09-11T08:04:22.981 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-09-11T08:04:22.981 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-09-11T08:04:22.981 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-09-11T08:04:22.981 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-09-11T08:04:22.981 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: 2024-09-11T08:04:22.981 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-09-11T08:04:22.981 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: seed 80691 2024-09-11T08:04:22.981 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-09-11T08:04:22.981 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (2 ms) 2024-09-11T08:04:22.981 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-09-11T08:04:22.981 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (278 ms) 2024-09-11T08:04:22.982 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-09-11T08:04:22.982 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (8 ms) 2024-09-11T08:04:22.982 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-09-11T08:04:22.982 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (8 ms) 2024-09-11T08:04:22.982 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-09-11T08:04:22.982 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (6 ms) 2024-09-11T08:04:22.982 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-09-11T08:04:22.982 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (2 ms) 2024-09-11T08:04:22.982 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-09-11T08:04:22.982 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (2 ms) 2024-09-11T08:04:22.982 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-09-11T08:04:22.982 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (7 ms) 2024-09-11T08:04:22.982 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-09-11T08:04:22.982 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (2 ms) 2024-09-11T08:04:22.983 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-09-11T08:04:22.983 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (9 ms) 2024-09-11T08:04:22.983 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-09-11T08:04:22.983 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (2 ms) 2024-09-11T08:04:22.983 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-09-11T08:04:22.983 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (6 ms) 2024-09-11T08:04:22.983 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-09-11T08:04:22.983 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (8 ms) 2024-09-11T08:04:22.983 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-09-11T08:04:22.983 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: osd_max_attr_size = 0 2024-09-11T08:04:22.983 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-09-11T08:04:22.983 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (1427 ms) 2024-09-11T08:04:22.983 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-09-11T08:04:22.984 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (490 ms) 2024-09-11T08:04:22.984 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-09-11T08:04:22.984 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-09-11T08:04:22.984 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: done waiting, doing copies 2024-09-11T08:04:22.984 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: waiting for final deep scrubs... 2024-09-11T08:04:22.984 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: done waiting 2024-09-11T08:04:22.984 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60603 ms) 2024-09-11T08:04:22.984 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-09-11T08:04:22.984 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (5 ms) 2024-09-11T08:04:22.984 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-09-11T08:04:22.984 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (3 ms) 2024-09-11T08:04:22.984 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-09-11T08:04:22.984 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4152 ms) 2024-09-11T08:04:22.984 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-09-11T08:04:22.984 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-09-11T08:04:22.985 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-09-11T08:04:22.985 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-09-11T08:04:22.985 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-09-11T08:04:22.985 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-09-11T08:04:23.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:23 smithi102 ceph-mon[41371]: pgmap v170: 332 pgs: 32 unknown, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:04:23.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:23.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:23 smithi102 ceph-mon[41371]: osdmap e135: 8 total, 8 up, 8 in 2024-09-11T08:04:23.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2526248987' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi102-80338-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:23.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:23 smithi102 ceph-mon[41371]: from='client.46619 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi102-80338-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:23.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80691-24"}]: dispatch 2024-09-11T08:04:23.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80691-24"}]: dispatch 2024-09-11T08:04:23.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi102-80691-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:23.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:23 smithi102 ceph-mon[29849]: pgmap v170: 332 pgs: 32 unknown, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:04:23.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:23.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:23 smithi102 ceph-mon[29849]: osdmap e135: 8 total, 8 up, 8 in 2024-09-11T08:04:23.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2526248987' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi102-80338-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:23.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:23 smithi102 ceph-mon[29849]: from='client.46619 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi102-80338-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:23.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80691-24"}]: dispatch 2024-09-11T08:04:23.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80691-24"}]: dispatch 2024-09-11T08:04:23.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi102-80691-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:23.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:23 smithi146 ceph-mon[33158]: pgmap v170: 332 pgs: 32 unknown, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:04:23.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:23.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:23 smithi146 ceph-mon[33158]: osdmap e135: 8 total, 8 up, 8 in 2024-09-11T08:04:23.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2526248987' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi102-80338-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:23.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:23 smithi146 ceph-mon[33158]: from='client.46619 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi102-80338-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:23.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80691-24"}]: dispatch 2024-09-11T08:04:23.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80691-24"}]: dispatch 2024-09-11T08:04:23.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi102-80691-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:24.588 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:24.589 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:24 smithi146 ceph-mon[33158]: from='client.46619 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi102-80338-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:24.589 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi102-80691-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:24.589 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:24 smithi146 ceph-mon[33158]: osdmap e136: 8 total, 8 up, 8 in 2024-09-11T08:04:24.589 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi102-80691-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi102-80691-24"}]: dispatch 2024-09-11T08:04:24.589 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2832693094' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi102-80356-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:24.589 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:24 smithi146 ceph-mon[33158]: from='client.47743 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi102-80356-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:24.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:24.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:24 smithi102 ceph-mon[41371]: from='client.46619 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi102-80338-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:24.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi102-80691-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:24.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:24 smithi102 ceph-mon[41371]: osdmap e136: 8 total, 8 up, 8 in 2024-09-11T08:04:24.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi102-80691-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi102-80691-24"}]: dispatch 2024-09-11T08:04:24.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2832693094' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi102-80356-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:24.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:24 smithi102 ceph-mon[41371]: from='client.47743 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi102-80356-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:24.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:24.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:24 smithi102 ceph-mon[29849]: from='client.46619 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi102-80338-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:24.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi102-80691-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:24.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:24 smithi102 ceph-mon[29849]: osdmap e136: 8 total, 8 up, 8 in 2024-09-11T08:04:24.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi102-80691-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi102-80691-24"}]: dispatch 2024-09-11T08:04:24.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2832693094' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi102-80356-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:24.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:24 smithi102 ceph-mon[29849]: from='client.47743 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi102-80356-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:24.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:04:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:04:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:04:25.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:25 smithi102 ceph-mon[41371]: pgmap v173: 332 pgs: 64 unknown, 268 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:04:25.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:25.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:25 smithi102 ceph-mon[41371]: from='client.47743 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi102-80356-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:25.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:25 smithi102 ceph-mon[41371]: osdmap e137: 8 total, 8 up, 8 in 2024-09-11T08:04:25.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:25 smithi102 ceph-mon[29849]: pgmap v173: 332 pgs: 64 unknown, 268 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:04:25.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:25.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:25 smithi102 ceph-mon[29849]: from='client.47743 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi102-80356-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:25.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:25 smithi102 ceph-mon[29849]: osdmap e137: 8 total, 8 up, 8 in 2024-09-11T08:04:25.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:25 smithi146 ceph-mon[33158]: pgmap v173: 332 pgs: 64 unknown, 268 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-11T08:04:25.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:25.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:25 smithi146 ceph-mon[33158]: from='client.47743 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi102-80356-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:25.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:25 smithi146 ceph-mon[33158]: osdmap e137: 8 total, 8 up, 8 in 2024-09-11T08:04:26.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:26.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi102-80691-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi102-80691-24"}]': finished 2024-09-11T08:04:26.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:26 smithi102 ceph-mon[41371]: osdmap e138: 8 total, 8 up, 8 in 2024-09-11T08:04:26.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2881767103' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi102-80338-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:26.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:26.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi102-80691-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi102-80691-24"}]': finished 2024-09-11T08:04:26.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:26 smithi102 ceph-mon[29849]: osdmap e138: 8 total, 8 up, 8 in 2024-09-11T08:04:26.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2881767103' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi102-80338-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:26.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:26.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi102-80691-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi102-80691-24"}]': finished 2024-09-11T08:04:26.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:26 smithi146 ceph-mon[33158]: osdmap e138: 8 total, 8 up, 8 in 2024-09-11T08:04:26.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2881767103' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi102-80338-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:27.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:27 smithi102 ceph-mon[41371]: pgmap v176: 308 pgs: 40 unknown, 268 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:04:27.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:27.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:27 smithi102 ceph-mon[41371]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:27.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2881767103' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi102-80338-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:27.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:27 smithi102 ceph-mon[41371]: osdmap e139: 8 total, 8 up, 8 in 2024-09-11T08:04:27.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3310394103' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi102-80356-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:27.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:27 smithi102 ceph-mon[41371]: from='client.46643 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi102-80356-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:27.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80691-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:27.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:27 smithi102 ceph-mon[29849]: pgmap v176: 308 pgs: 40 unknown, 268 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:04:27.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:27.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:27 smithi102 ceph-mon[29849]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:27.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2881767103' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi102-80338-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:27.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:27 smithi102 ceph-mon[29849]: osdmap e139: 8 total, 8 up, 8 in 2024-09-11T08:04:27.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3310394103' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi102-80356-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:27.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:27 smithi102 ceph-mon[29849]: from='client.46643 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi102-80356-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:27.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80691-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:27 smithi146 ceph-mon[33158]: pgmap v176: 308 pgs: 40 unknown, 268 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:04:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:27 smithi146 ceph-mon[33158]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2881767103' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi102-80338-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:27 smithi146 ceph-mon[33158]: osdmap e139: 8 total, 8 up, 8 in 2024-09-11T08:04:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3310394103' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi102-80356-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:27 smithi146 ceph-mon[33158]: from='client.46643 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi102-80356-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80691-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:28.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:28.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:28 smithi102 ceph-mon[41371]: from='client.46643 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi102-80356-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:28.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80691-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:28.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:28 smithi102 ceph-mon[41371]: osdmap e140: 8 total, 8 up, 8 in 2024-09-11T08:04:28.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80691-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:28.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1398843120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:28.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:28 smithi102 ceph-mon[41371]: from='client.46649 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:28.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1398843120' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:28.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:28 smithi102 ceph-mon[41371]: from='client.46649 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:28.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1398843120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi102-80338-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:28.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:28 smithi102 ceph-mon[41371]: from='client.46649 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi102-80338-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:28.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:28.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:28 smithi102 ceph-mon[29849]: from='client.46643 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi102-80356-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:28.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80691-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:28.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:28 smithi102 ceph-mon[29849]: osdmap e140: 8 total, 8 up, 8 in 2024-09-11T08:04:28.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80691-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:28.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1398843120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:28.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:28 smithi102 ceph-mon[29849]: from='client.46649 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:28.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1398843120' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:28.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:28 smithi102 ceph-mon[29849]: from='client.46649 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:28.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1398843120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi102-80338-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:28.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:28 smithi102 ceph-mon[29849]: from='client.46649 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi102-80338-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:28.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:28.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:28 smithi146 ceph-mon[33158]: from='client.46643 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi102-80356-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:28.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80691-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:28.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:28 smithi146 ceph-mon[33158]: osdmap e140: 8 total, 8 up, 8 in 2024-09-11T08:04:28.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80691-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:28.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1398843120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:28.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:28 smithi146 ceph-mon[33158]: from='client.46649 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:28.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1398843120' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:28.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:28 smithi146 ceph-mon[33158]: from='client.46649 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:28.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1398843120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi102-80338-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:28.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:28 smithi146 ceph-mon[33158]: from='client.46649 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi102-80338-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:29.849 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:29 smithi102 ceph-mon[29849]: pgmap v179: 340 pgs: 72 unknown, 268 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:04:29.849 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:29.849 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80691-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:29.849 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:29 smithi102 ceph-mon[29849]: from='client.46649 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi102-80338-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:29.849 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:29 smithi102 ceph-mon[29849]: osdmap e141: 8 total, 8 up, 8 in 2024-09-11T08:04:29.849 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1398843120' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi102-80338-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:29.849 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1275883381' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:29.849 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:29 smithi102 ceph-mon[29849]: from='client.46649 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi102-80338-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:29.849 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:29 smithi102 ceph-mon[29849]: from='client.46595 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:29.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:29 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:04:29.850 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:29 smithi102 ceph-mon[41371]: pgmap v179: 340 pgs: 72 unknown, 268 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:04:29.850 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:29.850 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80691-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:29.850 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:29 smithi102 ceph-mon[41371]: from='client.46649 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi102-80338-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:29.850 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:29 smithi102 ceph-mon[41371]: osdmap e141: 8 total, 8 up, 8 in 2024-09-11T08:04:29.850 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1398843120' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi102-80338-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:29.850 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1275883381' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:29.850 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:29 smithi102 ceph-mon[41371]: from='client.46649 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi102-80338-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:29.850 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:29 smithi102 ceph-mon[41371]: from='client.46595 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:29.850 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:29 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:04:29.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:04:29 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:04:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:29 smithi146 ceph-mon[33158]: pgmap v179: 340 pgs: 72 unknown, 268 active+clean; 592 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:04:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80691-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:29 smithi146 ceph-mon[33158]: from='client.46649 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi102-80338-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:29.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:29 smithi146 ceph-mon[33158]: osdmap e141: 8 total, 8 up, 8 in 2024-09-11T08:04:29.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1398843120' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi102-80338-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:29.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1275883381' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:29.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:29 smithi146 ceph-mon[33158]: from='client.46649 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi102-80338-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:29.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:29 smithi146 ceph-mon[33158]: from='client.46595 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:29.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:29 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:04:30.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:04:29 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:04:29] "GET /metrics HTTP/1.1" 200 235579 "" "Prometheus/2.43.0" 2024-09-11T08:04:30.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:30 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:04:30.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:30 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:30.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:30 smithi102 ceph-mon[41371]: from='client.46595 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]': finished 2024-09-11T08:04:30.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:30 smithi102 ceph-mon[41371]: osdmap e142: 8 total, 8 up, 8 in 2024-09-11T08:04:30.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:30 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1275883381' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:30.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:30 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/61410847' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi102-80356-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:30.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:30 smithi102 ceph-mon[41371]: from='client.46595 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:30.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:30 smithi102 ceph-mon[41371]: from='client.46655 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi102-80356-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:30.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:30 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:04:30.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:30 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:30.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:30 smithi102 ceph-mon[29849]: from='client.46595 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]': finished 2024-09-11T08:04:30.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:30 smithi102 ceph-mon[29849]: osdmap e142: 8 total, 8 up, 8 in 2024-09-11T08:04:30.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:30 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1275883381' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:30.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:30 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/61410847' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi102-80356-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:30.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:30 smithi102 ceph-mon[29849]: from='client.46595 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:30.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:30 smithi102 ceph-mon[29849]: from='client.46655 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi102-80356-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:30 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:04:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:30 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:30 smithi146 ceph-mon[33158]: from='client.46595 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi102-80697-15"}]': finished 2024-09-11T08:04:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:30 smithi146 ceph-mon[33158]: osdmap e142: 8 total, 8 up, 8 in 2024-09-11T08:04:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:30 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1275883381' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:30 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/61410847' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi102-80356-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:30 smithi146 ceph-mon[33158]: from='client.46595 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi102-80697-15"}]: dispatch 2024-09-11T08:04:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:30 smithi146 ceph-mon[33158]: from='client.46655 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi102-80356-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:31.019 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP (67 api_snapshots_pp: Running main() from gmock_main.cc 2024-09-11T08:04:31.019 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [==========] Running 20 tests from 5 test suites. 2024-09-11T08:04:31.020 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-09-11T08:04:31.020 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-09-11T08:04:31.020 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: seed 80697 2024-09-11T08:04:31.020 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-09-11T08:04:31.020 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (3077 ms) 2024-09-11T08:04:31.020 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-09-11T08:04:31.020 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (2737 ms) 2024-09-11T08:04:31.020 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-09-11T08:04:31.021 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (2520 ms) 2024-09-11T08:04:31.021 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-09-11T08:04:31.021 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2018 ms) 2024-09-11T08:04:31.021 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-09-11T08:04:31.021 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (4521 ms) 2024-09-11T08:04:31.021 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (14873 ms total) 2024-09-11T08:04:31.021 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: 2024-09-11T08:04:31.021 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [----------] 6 tests from LibRadosSnapshotsSelfManagedPP 2024-09-11T08:04:31.021 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-09-11T08:04:31.022 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4085 ms) 2024-09-11T08:04:31.022 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-09-11T08:04:31.022 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (4092 ms) 2024-09-11T08:04:31.022 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-09-11T08:04:31.022 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (5811 ms) 2024-09-11T08:04:31.022 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-09-11T08:04:31.022 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (4012 ms) 2024-09-11T08:04:31.022 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-09-11T08:04:31.022 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (1995 ms) 2024-09-11T08:04:31.022 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-09-11T08:04:31.022 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: deleting snap 23 in pool LibRadosSnapshotsSelfManagedPP_smithi102-80697-7 2024-09-11T08:04:31.022 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: waiting for snaps to purge 2024-09-11T08:04:31.022 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17980 ms) 2024-09-11T08:04:31.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [----------] 6 tests from LibRadosSnapshotsSelfManagedPP (37975 ms total) 2024-09-11T08:04:31.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: 2024-09-11T08:04:31.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-09-11T08:04:31.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-09-11T08:04:31.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (8 ms) 2024-09-11T08:04:31.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-09-11T08:04:31.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5064 ms) 2024-09-11T08:04:31.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5073 ms total) 2024-09-11T08:04:31.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: 2024-09-11T08:04:31.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-09-11T08:04:31.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-09-11T08:04:31.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (3001 ms) 2024-09-11T08:04:31.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-09-11T08:04:31.023 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapRemovePP (2004 ms) 2024-09-11T08:04:31.024 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-09-11T08:04:31.024 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2004 ms) 2024-09-11T08:04:31.024 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-09-11T08:04:31.024 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (2014 ms) 2024-09-11T08:04:31.024 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (9023 ms total) 2024-09-11T08:04:31.024 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: 2024-09-11T08:04:31.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[41371]: pgmap v182: 300 pgs: 32 creating+peering, 268 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:04:31.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:31.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[41371]: from='client.46649 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi102-80338-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi102-80338-26"}]': finished 2024-09-11T08:04:31.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[41371]: from='client.46595 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi102-80697-15"}]': finished 2024-09-11T08:04:31.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[41371]: from='client.46655 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi102-80356-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:31.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[41371]: osdmap e143: 8 total, 8 up, 8 in 2024-09-11T08:04:31.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80691-24"}]: dispatch 2024-09-11T08:04:31.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3955297845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:31.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[41371]: from='client.46661 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:31.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3955297845' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:31.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[41371]: from='client.46661 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:31.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3955297845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:31.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[41371]: from='client.46661 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:31.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[29849]: pgmap v182: 300 pgs: 32 creating+peering, 268 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:04:31.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:31.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[29849]: from='client.46649 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi102-80338-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi102-80338-26"}]': finished 2024-09-11T08:04:31.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[29849]: from='client.46595 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi102-80697-15"}]': finished 2024-09-11T08:04:31.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[29849]: from='client.46655 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi102-80356-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:31.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[29849]: osdmap e143: 8 total, 8 up, 8 in 2024-09-11T08:04:31.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80691-24"}]: dispatch 2024-09-11T08:04:31.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3955297845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:31.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[29849]: from='client.46661 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:31.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3955297845' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:31.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[29849]: from='client.46661 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:31.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3955297845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:31.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:31 smithi102 ceph-mon[29849]: from='client.46661 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:31 smithi146 ceph-mon[33158]: pgmap v182: 300 pgs: 32 creating+peering, 268 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:04:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:31 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:31 smithi146 ceph-mon[33158]: from='client.46649 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi102-80338-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi102-80338-26"}]': finished 2024-09-11T08:04:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:31 smithi146 ceph-mon[33158]: from='client.46595 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi102-80697-15"}]': finished 2024-09-11T08:04:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:31 smithi146 ceph-mon[33158]: from='client.46655 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi102-80356-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:31 smithi146 ceph-mon[33158]: osdmap e143: 8 total, 8 up, 8 in 2024-09-11T08:04:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:31 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80691-24"}]: dispatch 2024-09-11T08:04:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:31 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3955297845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:31 smithi146 ceph-mon[33158]: from='client.46661 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:31 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3955297845' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:31 smithi146 ceph-mon[33158]: from='client.46661 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:31 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3955297845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:31.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:31 smithi146 ceph-mon[33158]: from='client.46661 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:32.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:32.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80691-24"}]': finished 2024-09-11T08:04:32.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:32 smithi102 ceph-mon[41371]: from='client.46661 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:32.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:32 smithi102 ceph-mon[41371]: osdmap e144: 8 total, 8 up, 8 in 2024-09-11T08:04:32.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80691-24"}]: dispatch 2024-09-11T08:04:32.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3955297845' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:32.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:32 smithi102 ceph-mon[41371]: from='client.46661 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:32.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:32.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80691-24"}]': finished 2024-09-11T08:04:32.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:32 smithi102 ceph-mon[29849]: from='client.46661 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:32.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:32 smithi102 ceph-mon[29849]: osdmap e144: 8 total, 8 up, 8 in 2024-09-11T08:04:32.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80691-24"}]: dispatch 2024-09-11T08:04:32.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3955297845' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:32.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:32 smithi102 ceph-mon[29849]: from='client.46661 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:32.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:32.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80691-24"}]': finished 2024-09-11T08:04:32.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:32 smithi146 ceph-mon[33158]: from='client.46661 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:32.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:32 smithi146 ceph-mon[33158]: osdmap e144: 8 total, 8 up, 8 in 2024-09-11T08:04:32.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80691-24"}]: dispatch 2024-09-11T08:04:32.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3955297845' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:32.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:32 smithi146 ceph-mon[33158]: from='client.46661 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:33.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:33 smithi102 ceph-mon[29849]: pgmap v185: 268 pgs: 8 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:04:33.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:33.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80691-24"}]': finished 2024-09-11T08:04:33.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:33 smithi102 ceph-mon[29849]: osdmap e145: 8 total, 8 up, 8 in 2024-09-11T08:04:33.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1398843120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:33.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4082896817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi102-80356-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:33.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:33 smithi102 ceph-mon[29849]: from='client.46649 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:33.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:33 smithi102 ceph-mon[29849]: from='client.46667 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi102-80356-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:33.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:33 smithi102 ceph-mon[41371]: pgmap v185: 268 pgs: 8 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:04:33.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:33.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80691-24"}]': finished 2024-09-11T08:04:33.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:33 smithi102 ceph-mon[41371]: osdmap e145: 8 total, 8 up, 8 in 2024-09-11T08:04:33.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1398843120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:33.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4082896817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi102-80356-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:33.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:33 smithi102 ceph-mon[41371]: from='client.46649 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:33.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:33 smithi102 ceph-mon[41371]: from='client.46667 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi102-80356-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:33.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:33 smithi146 ceph-mon[33158]: pgmap v185: 268 pgs: 8 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:04:33.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:33.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2355360705' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80691-24"}]': finished 2024-09-11T08:04:33.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:33 smithi146 ceph-mon[33158]: osdmap e145: 8 total, 8 up, 8 in 2024-09-11T08:04:33.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1398843120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:33.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4082896817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi102-80356-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:33.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:33 smithi146 ceph-mon[33158]: from='client.46649 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:33.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:33 smithi146 ceph-mon[33158]: from='client.46667 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi102-80356-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:34.835 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:04:34 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:04:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:04:34.836 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:34 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:34.836 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:34 smithi146 ceph-mon[33158]: from='client.46661 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]': finished 2024-09-11T08:04:34.836 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:34 smithi146 ceph-mon[33158]: from='client.46649 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi102-80338-26"}]': finished 2024-09-11T08:04:34.836 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:34 smithi146 ceph-mon[33158]: from='client.46667 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi102-80356-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:34.836 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:34 smithi146 ceph-mon[33158]: osdmap e146: 8 total, 8 up, 8 in 2024-09-11T08:04:34.836 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:34 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1398843120' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:34.836 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:34 smithi146 ceph-mon[33158]: from='client.46649 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:34.836 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:34 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3969509726' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi102-80691-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:34.836 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:34 smithi146 ceph-mon[33158]: from='client.46673 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi102-80691-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:34.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:34 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:34.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:34 smithi102 ceph-mon[41371]: from='client.46661 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]': finished 2024-09-11T08:04:34.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:34 smithi102 ceph-mon[41371]: from='client.46649 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi102-80338-26"}]': finished 2024-09-11T08:04:34.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:34 smithi102 ceph-mon[41371]: from='client.46667 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi102-80356-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:34.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:34 smithi102 ceph-mon[41371]: osdmap e146: 8 total, 8 up, 8 in 2024-09-11T08:04:34.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:34 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1398843120' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:34.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:34 smithi102 ceph-mon[41371]: from='client.46649 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:34.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:34 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3969509726' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi102-80691-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:34.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:34 smithi102 ceph-mon[41371]: from='client.46673 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi102-80691-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:34.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:34 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:34.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:34 smithi102 ceph-mon[29849]: from='client.46661 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]': finished 2024-09-11T08:04:34.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:34 smithi102 ceph-mon[29849]: from='client.46649 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi102-80338-26"}]': finished 2024-09-11T08:04:34.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:34 smithi102 ceph-mon[29849]: from='client.46667 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi102-80356-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:34.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:34 smithi102 ceph-mon[29849]: osdmap e146: 8 total, 8 up, 8 in 2024-09-11T08:04:34.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:34 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1398843120' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:34.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:34 smithi102 ceph-mon[29849]: from='client.46649 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi102-80338-26"}]: dispatch 2024-09-11T08:04:34.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:34 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3969509726' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi102-80691-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:34.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:34 smithi102 ceph-mon[29849]: from='client.46673 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi102-80691-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:35.044 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp api_aio: Running main() from gmock_main.cc 2024-09-11T08:04:35.045 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [==========] Running 41 tests from 2 test suites. 2024-09-11T08:04:35.045 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [----------] Global test environment set-up. 2024-09-11T08:04:35.045 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [----------] 25 tests from LibRadosAio 2024-09-11T08:04:35.045 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-09-11T08:04:35.045 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.TooBig (6824 ms) 2024-09-11T08:04:35.045 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-09-11T08:04:35.045 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (3077 ms) 2024-09-11T08:04:35.045 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-09-11T08:04:35.045 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (3244 ms) 2024-09-11T08:04:35.046 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-09-11T08:04:35.046 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (3020 ms) 2024-09-11T08:04:35.046 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-09-11T08:04:35.046 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (3527 ms) 2024-09-11T08:04:35.046 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-09-11T08:04:35.046 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (3055 ms) 2024-09-11T08:04:35.046 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-09-11T08:04:35.046 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (3060 ms) 2024-09-11T08:04:35.046 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-09-11T08:04:35.046 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (2968 ms) 2024-09-11T08:04:35.046 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-09-11T08:04:35.046 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (3491 ms) 2024-09-11T08:04:35.046 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-09-11T08:04:35.047 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (2508 ms) 2024-09-11T08:04:35.047 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-09-11T08:04:35.047 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (2986 ms) 2024-09-11T08:04:35.047 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-09-11T08:04:35.047 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (3010 ms) 2024-09-11T08:04:35.047 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-09-11T08:04:35.047 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (2972 ms) 2024-09-11T08:04:35.047 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-09-11T08:04:35.047 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (3261 ms) 2024-09-11T08:04:35.047 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-09-11T08:04:35.048 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.Flush (2763 ms) 2024-09-11T08:04:35.048 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-09-11T08:04:35.048 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (3004 ms) 2024-09-11T08:04:35.048 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-09-11T08:04:35.048 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (2978 ms) 2024-09-11T08:04:35.048 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-09-11T08:04:35.048 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (3021 ms) 2024-09-11T08:04:35.048 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-09-11T08:04:35.048 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3018 ms) 2024-09-11T08:04:35.048 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-09-11T08:04:35.048 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3014 ms) 2024-09-11T08:04:35.048 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-09-11T08:04:35.049 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3018 ms) 2024-09-11T08:04:35.049 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-09-11T08:04:35.049 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (3021 ms) 2024-09-11T08:04:35.049 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-09-11T08:04:35.049 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3002 ms) 2024-09-11T08:04:35.049 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-09-11T08:04:35.049 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (3248 ms) 2024-09-11T08:04:35.049 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-09-11T08:04:35.049 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (2772 ms) 2024-09-11T08:04:35.049 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [----------] 25 tests from LibRadosAio (79862 ms total) 2024-09-11T08:04:35.049 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: 2024-09-11T08:04:35.049 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-09-11T08:04:35.050 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-09-11T08:04:35.050 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7045 ms) 2024-09-11T08:04:35.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[41371]: pgmap v188: 332 pgs: 72 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:04:35.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:35.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[41371]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:35.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[41371]: from='client.46649 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi102-80338-26"}]': finished 2024-09-11T08:04:35.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[41371]: from='client.46673 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi102-80691-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:35.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[41371]: osdmap e147: 8 total, 8 up, 8 in 2024-09-11T08:04:35.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2300941662' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:35.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[41371]: from='client.47791 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:35.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2300941662' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:35.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[41371]: from='client.47791 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:35.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2300941662' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi102-80338-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:35.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[41371]: from='client.47791 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi102-80338-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:35.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[29849]: pgmap v188: 332 pgs: 72 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:04:35.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:35.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[29849]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:35.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[29849]: from='client.46649 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi102-80338-26"}]': finished 2024-09-11T08:04:35.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[29849]: from='client.46673 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi102-80691-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:35.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[29849]: osdmap e147: 8 total, 8 up, 8 in 2024-09-11T08:04:35.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2300941662' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:35.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[29849]: from='client.47791 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:35.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2300941662' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:35.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[29849]: from='client.47791 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:35.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2300941662' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi102-80338-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:35.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:35 smithi102 ceph-mon[29849]: from='client.47791 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi102-80338-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:35.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:35 smithi146 ceph-mon[33158]: pgmap v188: 332 pgs: 72 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:04:35.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:35.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:35 smithi146 ceph-mon[33158]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:35.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:35 smithi146 ceph-mon[33158]: from='client.46649 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi102-80338-26"}]': finished 2024-09-11T08:04:35.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:35 smithi146 ceph-mon[33158]: from='client.46673 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi102-80691-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:35.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:35 smithi146 ceph-mon[33158]: osdmap e147: 8 total, 8 up, 8 in 2024-09-11T08:04:35.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2300941662' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:35.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:35 smithi146 ceph-mon[33158]: from='client.47791 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:35.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2300941662' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:35.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:35 smithi146 ceph-mon[33158]: from='client.47791 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:35.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2300941662' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi102-80338-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:35.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:35 smithi146 ceph-mon[33158]: from='client.47791 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi102-80338-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:36.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:36 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:36.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:36 smithi102 ceph-mon[29849]: from='client.47791 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi102-80338-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:36.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:36 smithi102 ceph-mon[29849]: osdmap e148: 8 total, 8 up, 8 in 2024-09-11T08:04:36.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:36 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2300941662' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi102-80338-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:36.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:36 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2672216810' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi102-80356-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:36.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:36 smithi102 ceph-mon[29849]: from='client.47791 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi102-80338-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:36.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:36 smithi102 ceph-mon[29849]: from='client.47788 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi102-80356-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:36.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:36 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:36.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:36 smithi102 ceph-mon[41371]: from='client.47791 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi102-80338-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:36.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:36 smithi102 ceph-mon[41371]: osdmap e148: 8 total, 8 up, 8 in 2024-09-11T08:04:36.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:36 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2300941662' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi102-80338-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:36.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:36 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2672216810' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi102-80356-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:36.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:36 smithi102 ceph-mon[41371]: from='client.47791 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi102-80338-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:36.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:36 smithi102 ceph-mon[41371]: from='client.47788 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi102-80356-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:36.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:36 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:36.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:36 smithi146 ceph-mon[33158]: from='client.47791 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi102-80338-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:36.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:36 smithi146 ceph-mon[33158]: osdmap e148: 8 total, 8 up, 8 in 2024-09-11T08:04:36.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:36 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2300941662' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi102-80338-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:36.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:36 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2672216810' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi102-80356-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:36.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:36 smithi146 ceph-mon[33158]: from='client.47791 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi102-80338-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:36.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:36 smithi146 ceph-mon[33158]: from='client.47788 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi102-80356-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:37.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:37 smithi102 ceph-mon[41371]: pgmap v191: 300 pgs: 32 unknown, 268 active+clean; 600 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-09-11T08:04:37.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:37.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:37 smithi102 ceph-mon[41371]: from='client.47788 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi102-80356-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:37.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:37 smithi102 ceph-mon[41371]: osdmap e149: 8 total, 8 up, 8 in 2024-09-11T08:04:37.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2296783824' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi102-80691-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:37.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:37 smithi102 ceph-mon[29849]: pgmap v191: 300 pgs: 32 unknown, 268 active+clean; 600 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-09-11T08:04:37.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:37.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:37 smithi102 ceph-mon[29849]: from='client.47788 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi102-80356-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:37.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:37 smithi102 ceph-mon[29849]: osdmap e149: 8 total, 8 up, 8 in 2024-09-11T08:04:37.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2296783824' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi102-80691-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:37.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:37 smithi146 ceph-mon[33158]: pgmap v191: 300 pgs: 32 unknown, 268 active+clean; 600 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-09-11T08:04:37.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:37.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:37 smithi146 ceph-mon[33158]: from='client.47788 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi102-80356-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:37.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:37 smithi146 ceph-mon[33158]: osdmap e149: 8 total, 8 up, 8 in 2024-09-11T08:04:37.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2296783824' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi102-80691-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:38.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:38.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:38 smithi102 ceph-mon[29849]: from='client.47791 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi102-80338-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi102-80338-27"}]': finished 2024-09-11T08:04:38.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2296783824' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi102-80691-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:38.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:38 smithi102 ceph-mon[29849]: osdmap e150: 8 total, 8 up, 8 in 2024-09-11T08:04:38.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:38 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:04:38.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:38.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:38 smithi102 ceph-mon[41371]: from='client.47791 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi102-80338-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi102-80338-27"}]': finished 2024-09-11T08:04:38.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2296783824' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi102-80691-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:38.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:38 smithi102 ceph-mon[41371]: osdmap e150: 8 total, 8 up, 8 in 2024-09-11T08:04:38.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:38 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:04:38.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:38.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:38 smithi146 ceph-mon[33158]: from='client.47791 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi102-80338-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi102-80338-27"}]': finished 2024-09-11T08:04:38.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2296783824' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi102-80691-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:38.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:38 smithi146 ceph-mon[33158]: osdmap e150: 8 total, 8 up, 8 in 2024-09-11T08:04:38.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:38 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:04:39.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:39 smithi102 ceph-mon[41371]: pgmap v194: 308 pgs: 40 unknown, 268 active+clean; 600 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-09-11T08:04:39.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:39 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:39.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:39 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:04:39.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:39 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:04:39.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:39 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:39.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:39 smithi102 ceph-mon[41371]: osdmap e151: 8 total, 8 up, 8 in 2024-09-11T08:04:39.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:39 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1616985714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi102-80356-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:39.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:39 smithi102 ceph-mon[41371]: from='client.46691 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi102-80356-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:39.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:39 smithi102 ceph-mon[29849]: pgmap v194: 308 pgs: 40 unknown, 268 active+clean; 600 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-09-11T08:04:39.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:39 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:39.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:39 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:04:39.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:39 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:04:39.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:39 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:39.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:39 smithi102 ceph-mon[29849]: osdmap e151: 8 total, 8 up, 8 in 2024-09-11T08:04:39.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:39 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1616985714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi102-80356-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:39.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:39 smithi102 ceph-mon[29849]: from='client.46691 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi102-80356-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:39.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:04:39 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:04:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:39 smithi146 ceph-mon[33158]: pgmap v194: 308 pgs: 40 unknown, 268 active+clean; 600 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-09-11T08:04:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:39 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:04:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:39 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:04:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:39 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:04:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:39 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:39 smithi146 ceph-mon[33158]: osdmap e151: 8 total, 8 up, 8 in 2024-09-11T08:04:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:39 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1616985714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi102-80356-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:39 smithi146 ceph-mon[33158]: from='client.46691 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi102-80356-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:40.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:04:39 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:04:39] "GET /metrics HTTP/1.1" 200 235579 "" "Prometheus/2.43.0" 2024-09-11T08:04:40.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:40 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:04:40.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:40.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:40 smithi102 ceph-mon[29849]: from='client.46691 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi102-80356-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:40.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:40 smithi102 ceph-mon[29849]: osdmap e152: 8 total, 8 up, 8 in 2024-09-11T08:04:40.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2596589619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi102-80691-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:40.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:40 smithi102 ceph-mon[29849]: from='client.46697 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi102-80691-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:40.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2300941662' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:40.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:40 smithi102 ceph-mon[29849]: from='client.47791 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:40.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:04:40.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:40 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:04:40.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:40 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:04:40.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:40.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:40 smithi102 ceph-mon[41371]: from='client.46691 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi102-80356-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:40.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:40 smithi102 ceph-mon[41371]: osdmap e152: 8 total, 8 up, 8 in 2024-09-11T08:04:40.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2596589619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi102-80691-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:40.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:40 smithi102 ceph-mon[41371]: from='client.46697 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi102-80691-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:40.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2300941662' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:40.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:40 smithi102 ceph-mon[41371]: from='client.47791 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:40.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:04:40.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:40 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:04:40.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:40 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:04:40.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:40.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:40 smithi146 ceph-mon[33158]: from='client.46691 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi102-80356-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:40.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:40 smithi146 ceph-mon[33158]: osdmap e152: 8 total, 8 up, 8 in 2024-09-11T08:04:40.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2596589619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi102-80691-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:40.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:40 smithi146 ceph-mon[33158]: from='client.46697 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi102-80691-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:40.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2300941662' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:40.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:40 smithi146 ceph-mon[33158]: from='client.47791 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:40.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:04:40.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:40 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:04:41.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:41 smithi102 ceph-mon[29849]: pgmap v197: 332 pgs: 32 creating+peering, 300 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 24 op/s 2024-09-11T08:04:41.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:41.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:41 smithi102 ceph-mon[29849]: from='client.46697 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi102-80691-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:41.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:41 smithi102 ceph-mon[29849]: from='client.47791 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi102-80338-27"}]': finished 2024-09-11T08:04:41.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:41 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:04:41.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:41 smithi102 ceph-mon[29849]: osdmap e153: 8 total, 8 up, 8 in 2024-09-11T08:04:41.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2300941662' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:41.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:41 smithi102 ceph-mon[29849]: from='client.47791 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:41.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-13"}]: dispatch 2024-09-11T08:04:41.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:41 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-13"}]: dispatch 2024-09-11T08:04:41.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:41 smithi102 ceph-mon[41371]: pgmap v197: 332 pgs: 32 creating+peering, 300 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 24 op/s 2024-09-11T08:04:41.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:41.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:41 smithi102 ceph-mon[41371]: from='client.46697 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi102-80691-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:41.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:41 smithi102 ceph-mon[41371]: from='client.47791 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi102-80338-27"}]': finished 2024-09-11T08:04:41.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:41 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:04:41.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:41 smithi102 ceph-mon[41371]: osdmap e153: 8 total, 8 up, 8 in 2024-09-11T08:04:41.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2300941662' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:41.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:41 smithi102 ceph-mon[41371]: from='client.47791 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:41.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-13"}]: dispatch 2024-09-11T08:04:41.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:41 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-13"}]: dispatch 2024-09-11T08:04:41.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:41 smithi146 ceph-mon[33158]: pgmap v197: 332 pgs: 32 creating+peering, 300 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 24 op/s 2024-09-11T08:04:41.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:41.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:41 smithi146 ceph-mon[33158]: from='client.46697 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi102-80691-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:41.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:41 smithi146 ceph-mon[33158]: from='client.47791 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi102-80338-27"}]': finished 2024-09-11T08:04:41.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:41 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:04:41.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:41 smithi146 ceph-mon[33158]: osdmap e153: 8 total, 8 up, 8 in 2024-09-11T08:04:41.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2300941662' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:41.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:41 smithi146 ceph-mon[33158]: from='client.47791 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi102-80338-27"}]: dispatch 2024-09-11T08:04:41.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-13"}]: dispatch 2024-09-11T08:04:41.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:41 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-13"}]: dispatch 2024-09-11T08:04:42.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:04:42.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[41371]: from='client.47791 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi102-80338-27"}]': finished 2024-09-11T08:04:42.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-13"}]': finished 2024-09-11T08:04:42.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[41371]: osdmap e154: 8 total, 8 up, 8 in 2024-09-11T08:04:42.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2729351943' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi102-80356-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:42.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi102-80691-36"}]: dispatch 2024-09-11T08:04:42.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi102-80691-36"}]: dispatch 2024-09-11T08:04:42.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi102-80691-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:42.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3632195509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:42.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[41371]: from='client.46715 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:42.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3632195509' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:42.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[41371]: from='client.46715 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:42.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3632195509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi102-80338-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:42.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[41371]: from='client.46715 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi102-80338-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:42.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:42.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:04:42.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[29849]: from='client.47791 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi102-80338-27"}]': finished 2024-09-11T08:04:42.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-13"}]': finished 2024-09-11T08:04:42.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[29849]: osdmap e154: 8 total, 8 up, 8 in 2024-09-11T08:04:42.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2729351943' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi102-80356-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:42.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi102-80691-36"}]: dispatch 2024-09-11T08:04:42.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi102-80691-36"}]: dispatch 2024-09-11T08:04:42.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi102-80691-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:42.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3632195509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:42.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[29849]: from='client.46715 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:42.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3632195509' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:42.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[29849]: from='client.46715 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:42.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3632195509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi102-80338-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:42.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[29849]: from='client.46715 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi102-80338-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:42.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:42.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:42 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:04:42.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:42 smithi146 ceph-mon[33158]: from='client.47791 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi102-80338-27"}]': finished 2024-09-11T08:04:42.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:42 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-13"}]': finished 2024-09-11T08:04:42.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:42 smithi146 ceph-mon[33158]: osdmap e154: 8 total, 8 up, 8 in 2024-09-11T08:04:42.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2729351943' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi102-80356-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:42.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi102-80691-36"}]: dispatch 2024-09-11T08:04:42.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi102-80691-36"}]: dispatch 2024-09-11T08:04:42.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi102-80691-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:42.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3632195509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:42.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:42 smithi146 ceph-mon[33158]: from='client.46715 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:42.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3632195509' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:42.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:42 smithi146 ceph-mon[33158]: from='client.46715 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:42.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3632195509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi102-80338-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:42.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:42 smithi146 ceph-mon[33158]: from='client.46715 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi102-80338-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:42.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:44.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:44 smithi102 ceph-mon[29849]: pgmap v200: 300 pgs: 32 unknown, 268 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 24 op/s 2024-09-11T08:04:44.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:44 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2729351943' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi102-80356-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:44.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:44 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi102-80691-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:44.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:44 smithi102 ceph-mon[29849]: from='client.46715 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi102-80338-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:44.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:44 smithi102 ceph-mon[29849]: osdmap e155: 8 total, 8 up, 8 in 2024-09-11T08:04:44.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:44 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi102-80691-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi102-80691-36"}]: dispatch 2024-09-11T08:04:44.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:44 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3632195509' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi102-80338-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:44.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:44 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:44.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:44 smithi102 ceph-mon[29849]: from='client.46715 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi102-80338-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:44.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:44 smithi102 ceph-mon[41371]: pgmap v200: 300 pgs: 32 unknown, 268 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 24 op/s 2024-09-11T08:04:44.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:44 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2729351943' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi102-80356-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:44.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:44 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi102-80691-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:44.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:44 smithi102 ceph-mon[41371]: from='client.46715 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi102-80338-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:44.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:44 smithi102 ceph-mon[41371]: osdmap e155: 8 total, 8 up, 8 in 2024-09-11T08:04:44.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:44 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi102-80691-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi102-80691-36"}]: dispatch 2024-09-11T08:04:44.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:44 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3632195509' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi102-80338-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:44.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:44 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:44.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:44 smithi102 ceph-mon[41371]: from='client.46715 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi102-80338-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:44.684 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:04:44 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:04:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:04:44.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:44 smithi146 ceph-mon[33158]: pgmap v200: 300 pgs: 32 unknown, 268 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 24 op/s 2024-09-11T08:04:44.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:44 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2729351943' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi102-80356-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:44.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:44 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi102-80691-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:44.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:44 smithi146 ceph-mon[33158]: from='client.46715 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi102-80338-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:44.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:44 smithi146 ceph-mon[33158]: osdmap e155: 8 total, 8 up, 8 in 2024-09-11T08:04:44.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:44 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi102-80691-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi102-80691-36"}]: dispatch 2024-09-11T08:04:44.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:44 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3632195509' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi102-80338-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:44.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:44 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:44.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:44 smithi146 ceph-mon[33158]: from='client.46715 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi102-80338-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:45.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:45.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:45.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[29849]: osdmap e156: 8 total, 8 up, 8 in 2024-09-11T08:04:45.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:45.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:04:45.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:04:45.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[29849]: pgmap v203: 268 pgs: 32 unknown, 236 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:04:45.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi102-80691-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi102-80691-36"}]': finished 2024-09-11T08:04:45.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[29849]: from='client.46715 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi102-80338-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi102-80338-28"}]': finished 2024-09-11T08:04:45.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:45.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[29849]: osdmap e157: 8 total, 8 up, 8 in 2024-09-11T08:04:45.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1238762683' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi102-80356-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:45.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:45.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:45.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:45.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[41371]: osdmap e156: 8 total, 8 up, 8 in 2024-09-11T08:04:45.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:45.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:04:45.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:04:45.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[41371]: pgmap v203: 268 pgs: 32 unknown, 236 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:04:45.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi102-80691-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi102-80691-36"}]': finished 2024-09-11T08:04:45.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[41371]: from='client.46715 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi102-80338-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi102-80338-28"}]': finished 2024-09-11T08:04:45.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:45.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[41371]: osdmap e157: 8 total, 8 up, 8 in 2024-09-11T08:04:45.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1238762683' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi102-80356-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:45.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:45 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:45.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:45 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:45.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:45 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:45.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:45 smithi146 ceph-mon[33158]: osdmap e156: 8 total, 8 up, 8 in 2024-09-11T08:04:45.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:45 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:45.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:45 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:04:45.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:45 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:04:45.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:45 smithi146 ceph-mon[33158]: pgmap v203: 268 pgs: 32 unknown, 236 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:04:45.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:45 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi102-80691-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi102-80691-36"}]': finished 2024-09-11T08:04:45.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:45 smithi146 ceph-mon[33158]: from='client.46715 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi102-80338-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi102-80338-28"}]': finished 2024-09-11T08:04:45.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:45 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:45.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:45 smithi146 ceph-mon[33158]: osdmap e157: 8 total, 8 up, 8 in 2024-09-11T08:04:45.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:45 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1238762683' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi102-80356-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:45.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:45 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:47.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:46 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1238762683' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi102-80356-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:47.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:46 smithi146 ceph-mon[33158]: osdmap e158: 8 total, 8 up, 8 in 2024-09-11T08:04:47.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:46 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:04:47.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:46 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:04:47.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:46 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:47.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1238762683' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi102-80356-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:47.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:46 smithi102 ceph-mon[29849]: osdmap e158: 8 total, 8 up, 8 in 2024-09-11T08:04:47.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:04:47.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:46 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:04:47.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:47.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:46 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1238762683' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi102-80356-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:47.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:46 smithi102 ceph-mon[41371]: osdmap e158: 8 total, 8 up, 8 in 2024-09-11T08:04:47.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:46 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:04:47.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:46 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:04:47.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:46 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:47 smithi146 ceph-mon[33158]: pgmap v206: 316 pgs: 48 creating+peering, 268 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:04:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:47 smithi146 ceph-mon[33158]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:47 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:04:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:47 smithi146 ceph-mon[33158]: osdmap e159: 8 total, 8 up, 8 in 2024-09-11T08:04:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-15"}]: dispatch 2024-09-11T08:04:48.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:47 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-15"}]: dispatch 2024-09-11T08:04:48.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3955297845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:48.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3632195509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:48.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi102-80691-36"}]: dispatch 2024-09-11T08:04:48.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:47 smithi146 ceph-mon[33158]: from='client.46661 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:48.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:47 smithi146 ceph-mon[33158]: from='client.46715 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:48.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:48.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[29849]: pgmap v206: 316 pgs: 48 creating+peering, 268 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:04:48.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[29849]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:48.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:04:48.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[29849]: osdmap e159: 8 total, 8 up, 8 in 2024-09-11T08:04:48.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-15"}]: dispatch 2024-09-11T08:04:48.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-15"}]: dispatch 2024-09-11T08:04:48.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3955297845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:48.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3632195509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:48.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi102-80691-36"}]: dispatch 2024-09-11T08:04:48.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[29849]: from='client.46661 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:48.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[29849]: from='client.46715 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:48.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:48.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[41371]: pgmap v206: 316 pgs: 48 creating+peering, 268 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:04:48.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[41371]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:48.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:04:48.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[41371]: osdmap e159: 8 total, 8 up, 8 in 2024-09-11T08:04:48.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-15"}]: dispatch 2024-09-11T08:04:48.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-15"}]: dispatch 2024-09-11T08:04:48.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3955297845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:48.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3632195509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:48.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi102-80691-36"}]: dispatch 2024-09-11T08:04:48.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[41371]: from='client.46661 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:48.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[41371]: from='client.46715 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:48.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:48.934 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComple020 ms total) 2024-09-11T08:04:48.935 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: 2024-09-11T08:04:48.935 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-09-11T08:04:48.935 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-09-11T08:04:48.935 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (303 ms) 2024-09-11T08:04:48.935 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (303 ms total) 2024-09-11T08:04:48.935 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: 2024-09-11T08:04:48.935 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-09-11T08:04:48.935 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-09-11T08:04:48.935 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (23 ms) 2024-09-11T08:04:48.935 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-09-11T08:04:48.936 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (9 ms) 2024-09-11T08:04:48.936 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (32 ms total) 2024-09-11T08:04:48.936 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: 2024-09-11T08:04:48.936 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-09-11T08:04:48.936 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-09-11T08:04:48.936 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (31 ms) 2024-09-11T08:04:48.936 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-09-11T08:04:48.936 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (13 ms) 2024-09-11T08:04:48.936 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (44 ms total) 2024-09-11T08:04:48.936 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: 2024-09-11T08:04:48.936 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-09-11T08:04:48.936 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-09-11T08:04:48.936 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (28 ms) 2024-09-11T08:04:48.936 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-09-11T08:04:48.937 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (5 ms) 2024-09-11T08:04:48.937 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (33 ms total) 2024-09-11T08:04:48.937 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: 2024-09-11T08:04:48.937 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-09-11T08:04:48.937 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-09-11T08:04:48.937 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1033 ms) 2024-09-11T08:04:48.937 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1033 ms total) 2024-09-11T08:04:48.937 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: 2024-09-11T08:04:48.937 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-09-11T08:04:48.937 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (100677 ms total) 2024-09-11T08:04:48.938 INFO:tasks.workunit.client.0.smithi102.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-09-11T08:04:48.938 INFO:tasks.workunit.client.0.smithi102.stdout:: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-09-11T08:04:48.938 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-09-11T08:04:48.938 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4060 ms) 2024-09-11T08:04:48.938 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-09-11T08:04:48.939 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3794 ms) 2024-09-11T08:04:48.939 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-09-11T08:04:48.939 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (4017 ms) 2024-09-11T08:04:48.939 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (11871 ms total) 2024-09-11T08:04:48.939 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: 2024-09-11T08:04:48.939 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-09-11T08:04:48.939 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [==========] 20 tests from 5 test suites ran. (100694 ms total) 2024-09-11T08:04:48.939 INFO:tasks.workunit.client.0.smithi102.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-09-11T08:04:49.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:48 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-15"}]': finished 2024-09-11T08:04:49.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi102-80691-36"}]': finished 2024-09-11T08:04:49.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:48 smithi146 ceph-mon[33158]: from='client.46661 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]': finished 2024-09-11T08:04:49.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:48 smithi146 ceph-mon[33158]: from='client.46715 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi102-80338-28"}]': finished 2024-09-11T08:04:49.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:48 smithi146 ceph-mon[33158]: osdmap e160: 8 total, 8 up, 8 in 2024-09-11T08:04:49.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-15", "mode": "writeback"}]: dispatch 2024-09-11T08:04:49.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi102-80691-36"}]: dispatch 2024-09-11T08:04:49.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:48 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-15", "mode": "writeback"}]: dispatch 2024-09-11T08:04:49.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3632195509' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:49.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2271723120' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi102-80356-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:49.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3955297845' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:49.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:48 smithi146 ceph-mon[33158]: from='client.46715 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:49.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:48 smithi146 ceph-mon[33158]: from='client.46727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi102-80356-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:49.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:48 smithi146 ceph-mon[33158]: from='client.46661 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:49.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:49.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-15"}]': finished 2024-09-11T08:04:49.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi102-80691-36"}]': finished 2024-09-11T08:04:49.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[29849]: from='client.46661 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]': finished 2024-09-11T08:04:49.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[29849]: from='client.46715 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi102-80338-28"}]': finished 2024-09-11T08:04:49.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[29849]: osdmap e160: 8 total, 8 up, 8 in 2024-09-11T08:04:49.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-15", "mode": "writeback"}]: dispatch 2024-09-11T08:04:49.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi102-80691-36"}]: dispatch 2024-09-11T08:04:49.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-15", "mode": "writeback"}]: dispatch 2024-09-11T08:04:49.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3632195509' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:49.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2271723120' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi102-80356-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:49.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3955297845' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:49.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[29849]: from='client.46715 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:49.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[29849]: from='client.46727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi102-80356-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:49.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[29849]: from='client.46661 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:49.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:49.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-15"}]': finished 2024-09-11T08:04:49.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi102-80691-36"}]': finished 2024-09-11T08:04:49.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[41371]: from='client.46661 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]': finished 2024-09-11T08:04:49.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[41371]: from='client.46715 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi102-80338-28"}]': finished 2024-09-11T08:04:49.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[41371]: osdmap e160: 8 total, 8 up, 8 in 2024-09-11T08:04:49.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-15", "mode": "writeback"}]: dispatch 2024-09-11T08:04:49.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi102-80691-36"}]: dispatch 2024-09-11T08:04:49.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-15", "mode": "writeback"}]: dispatch 2024-09-11T08:04:49.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3632195509' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:49.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2271723120' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi102-80356-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:49.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3955297845' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:49.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[41371]: from='client.46715 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi102-80338-28"}]: dispatch 2024-09-11T08:04:49.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[41371]: from='client.46727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi102-80356-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:49.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[41371]: from='client.46661 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]: dispatch 2024-09-11T08:04:49.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:49.923 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:04:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:04:50.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:04:49] "GET /metrics HTTP/1.1" 200 233254 "" "Prometheus/2.43.0" 2024-09-11T08:04:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[29849]: pgmap v209: 292 pgs: 32 unknown, 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:04:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:04:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi102-80691-36"}]': finished 2024-09-11T08:04:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-15", "mode": "writeback"}]': finished 2024-09-11T08:04:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[29849]: from='client.46715 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi102-80338-28"}]': finished 2024-09-11T08:04:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[29849]: from='client.46727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi102-80356-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[29849]: from='client.46661 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]': finished 2024-09-11T08:04:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[29849]: osdmap e161: 8 total, 8 up, 8 in 2024-09-11T08:04:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3656025770' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[29849]: from='client.46733 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3656025770' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[29849]: from='client.46733 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:50.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3656025770' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi102-80338-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:50.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[29849]: from='client.46733 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi102-80338-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:50.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:50.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:04:50.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[41371]: pgmap v209: 292 pgs: 32 unknown, 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:04:50.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:04:50.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi102-80691-36"}]': finished 2024-09-11T08:04:50.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-15", "mode": "writeback"}]': finished 2024-09-11T08:04:50.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[41371]: from='client.46715 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi102-80338-28"}]': finished 2024-09-11T08:04:50.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[41371]: from='client.46727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi102-80356-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:50.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[41371]: from='client.46661 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]': finished 2024-09-11T08:04:50.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[41371]: osdmap e161: 8 total, 8 up, 8 in 2024-09-11T08:04:50.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3656025770' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:50.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[41371]: from='client.46733 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:50.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3656025770' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:50.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[41371]: from='client.46733 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:50.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3656025770' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi102-80338-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:50.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[41371]: from='client.46733 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi102-80338-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:50.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:50.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:49 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:04:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:49 smithi146 ceph-mon[33158]: pgmap v209: 292 pgs: 32 unknown, 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:04:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:49 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:04:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:49 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2178867550' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi102-80691-36"}]': finished 2024-09-11T08:04:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:49 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-15", "mode": "writeback"}]': finished 2024-09-11T08:04:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:49 smithi146 ceph-mon[33158]: from='client.46715 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi102-80338-28"}]': finished 2024-09-11T08:04:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:49 smithi146 ceph-mon[33158]: from='client.46727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi102-80356-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:49 smithi146 ceph-mon[33158]: from='client.46661 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi102-80697-20"}]': finished 2024-09-11T08:04:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:49 smithi146 ceph-mon[33158]: osdmap e161: 8 total, 8 up, 8 in 2024-09-11T08:04:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:49 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3656025770' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:49 smithi146 ceph-mon[33158]: from='client.46733 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:49 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3656025770' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:49 smithi146 ceph-mon[33158]: from='client.46733 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:50.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:49 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3656025770' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi102-80338-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:50.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:49 smithi146 ceph-mon[33158]: from='client.46733 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi102-80338-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:50.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:49 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:50.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:49 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:04:51.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:50 smithi146 ceph-mon[33158]: from='client.46733 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi102-80338-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:51.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:50 smithi146 ceph-mon[33158]: osdmap e162: 8 total, 8 up, 8 in 2024-09-11T08:04:51.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:50 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3656025770' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi102-80338-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:51.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:50 smithi146 ceph-mon[33158]: from='client.46733 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi102-80338-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:51.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:50 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:04:51.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:50 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:51.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:50 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:04:51.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:50 smithi146 ceph-mon[33158]: pgmap v212: 260 pgs: 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:04:51.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:50 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi102-80411-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-11T08:04:51.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:50 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi102-80411-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-11T08:04:51.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:50 smithi102 ceph-mon[29849]: from='client.46733 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi102-80338-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:50 smithi102 ceph-mon[29849]: osdmap e162: 8 total, 8 up, 8 in 2024-09-11T08:04:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:50 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3656025770' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi102-80338-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:50 smithi102 ceph-mon[29849]: from='client.46733 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi102-80338-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:50 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:04:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:50 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:50 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:04:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:50 smithi102 ceph-mon[29849]: pgmap v212: 260 pgs: 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:04:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:50 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi102-80411-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-11T08:04:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:50 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi102-80411-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-11T08:04:51.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:50 smithi102 ceph-mon[41371]: from='client.46733 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi102-80338-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:51.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:50 smithi102 ceph-mon[41371]: osdmap e162: 8 total, 8 up, 8 in 2024-09-11T08:04:51.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:50 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3656025770' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi102-80338-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:51.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:50 smithi102 ceph-mon[41371]: from='client.46733 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi102-80338-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:51.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:50 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:04:51.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:50 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:51.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:50 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:04:51.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:50 smithi102 ceph-mon[41371]: pgmap v212: 260 pgs: 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:04:51.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:50 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi102-80411-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-11T08:04:51.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:50 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi102-80411-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-11T08:04:52.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:51 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:52.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:51 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:04:52.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:51 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi102-80411-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-11T08:04:52.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:51 smithi146 ceph-mon[33158]: osdmap e163: 8 total, 8 up, 8 in 2024-09-11T08:04:52.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:51 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-15"}]: dispatch 2024-09-11T08:04:52.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:51 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1954126667' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi102-80356-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:52.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:51 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-15"}]: dispatch 2024-09-11T08:04:52.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:51 smithi146 ceph-mon[33158]: from='client.46736 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi102-80356-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:52.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:51 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:04:52.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:51 smithi146 ceph-mon[33158]: from='client.46733 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi102-80338-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi102-80338-29"}]': finished 2024-09-11T08:04:52.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:51 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-15"}]': finished 2024-09-11T08:04:52.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:51 smithi146 ceph-mon[33158]: from='client.46736 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi102-80356-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:52.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:51 smithi146 ceph-mon[33158]: osdmap e164: 8 total, 8 up, 8 in 2024-09-11T08:04:52.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:52.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:04:52.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi102-80411-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-11T08:04:52.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[29849]: osdmap e163: 8 total, 8 up, 8 in 2024-09-11T08:04:52.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-15"}]: dispatch 2024-09-11T08:04:52.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1954126667' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi102-80356-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:52.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-15"}]: dispatch 2024-09-11T08:04:52.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[29849]: from='client.46736 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi102-80356-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:52.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:04:52.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[29849]: from='client.46733 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi102-80338-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi102-80338-29"}]': finished 2024-09-11T08:04:52.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-15"}]': finished 2024-09-11T08:04:52.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[29849]: from='client.46736 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi102-80356-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:52.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[29849]: osdmap e164: 8 total, 8 up, 8 in 2024-09-11T08:04:52.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:52.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:04:52.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi102-80411-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-11T08:04:52.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[41371]: osdmap e163: 8 total, 8 up, 8 in 2024-09-11T08:04:52.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-15"}]: dispatch 2024-09-11T08:04:52.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1954126667' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi102-80356-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:52.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-15"}]: dispatch 2024-09-11T08:04:52.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[41371]: from='client.46736 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi102-80356-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:52.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:04:52.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[41371]: from='client.46733 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi102-80338-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi102-80338-29"}]': finished 2024-09-11T08:04:52.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-15"}]': finished 2024-09-11T08:04:52.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[41371]: from='client.46736 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi102-80356-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:52.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:51 smithi102 ceph-mon[41371]: osdmap e164: 8 total, 8 up, 8 in 2024-09-11T08:04:52.889 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-09-11T08:04:52.889 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-09-11T08:04:52.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-09-11T08:04:52.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-09-11T08:04:52.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-09-11T08:04:52.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (6813 ms) 2024-09-11T08:04:52.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-09-11T08:04:52.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (18957 ms) 2024-09-11T08:04:52.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-09-11T08:04:52.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (6505 ms) 2024-09-11T08:04:52.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-09-11T08:04:52.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (3476 ms) 2024-09-11T08:04:52.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-09-11T08:04:52.890 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (3022 ms) 2024-09-11T08:04:52.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-09-11T08:04:52.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (2993 ms) 2024-09-11T08:04:52.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-09-11T08:04:52.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (2976 ms) 2024-09-11T08:04:52.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-09-11T08:04:52.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (3009 ms) 2024-09-11T08:04:52.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-09-11T08:04:52.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (3005 ms) 2024-09-11T08:04:52.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-09-11T08:04:52.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (2983 ms) 2024-09-11T08:04:52.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-09-11T08:04:52.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (3012 ms) 2024-09-11T08:04:52.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-09-11T08:04:52.891 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (3022 ms) 2024-09-11T08:04:52.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-09-11T08:04:52.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3014 ms) 2024-09-11T08:04:52.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-09-11T08:04:52.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3017 ms) 2024-09-11T08:04:52.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-09-11T08:04:52.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3028 ms) 2024-09-11T08:04:52.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-09-11T08:04:52.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3001 ms) 2024-09-11T08:04:52.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-09-11T08:04:52.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (3009 ms) 2024-09-11T08:04:52.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-09-11T08:04:52.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (3002 ms) 2024-09-11T08:04:52.892 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-09-11T08:04:52.893 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3024 ms) 2024-09-11T08:04:52.893 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-09-11T08:04:52.893 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3017 ms) 2024-09-11T08:04:52.893 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-09-11T08:04:52.893 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3014 ms) 2024-09-11T08:04:52.893 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-09-11T08:04:52.893 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3007 ms) 2024-09-11T08:04:52.893 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-09-11T08:04:52.893 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3016 ms) 2024-09-11T08:04:52.893 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-09-11T08:04:52.893 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (3304 ms) 2024-09-11T08:04:52.893 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-09-11T08:04:52.893 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (2538 ms) 2024-09-11T08:04:52.893 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-09-11T08:04:52.893 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (3018 ms) 2024-09-11T08:04:52.894 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-09-11T08:04:53.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:53 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:53.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:53 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:04:53.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:53 smithi102 ceph-mon[29849]: from='client.24692 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:04:53.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:53 smithi102 ceph-mon[29849]: from='client.24692 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pg_num","val":"11"}]': finished 2024-09-11T08:04:53.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:53 smithi102 ceph-mon[29849]: osdmap e165: 8 total, 8 up, 8 in 2024-09-11T08:04:53.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:53 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:53.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:53 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:53.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:53 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:04:53.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:53 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:53.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:53 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:04:53.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:53 smithi102 ceph-mon[41371]: from='client.24692 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:04:53.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:53 smithi102 ceph-mon[41371]: from='client.24692 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pg_num","val":"11"}]': finished 2024-09-11T08:04:53.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:53 smithi102 ceph-mon[41371]: osdmap e165: 8 total, 8 up, 8 in 2024-09-11T08:04:53.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:53 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:53.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:53 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:53.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:53 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:04:53.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:53 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:53.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:53 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:04:53.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:53 smithi146 ceph-mon[33158]: from='client.24692 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pg_num","val":"11"}]: dispatch 2024-09-11T08:04:53.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:53 smithi146 ceph-mon[33158]: from='client.24692 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pg_num","val":"11"}]': finished 2024-09-11T08:04:53.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:53 smithi146 ceph-mon[33158]: osdmap e165: 8 total, 8 up, 8 in 2024-09-11T08:04:53.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:53 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:53.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:53 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:53.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:53 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:04:54.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[41371]: pgmap v215: 300 pgs: 40 unknown, 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:04:54.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:54.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[41371]: from='client.24692 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:04:54.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[41371]: from='client.24692 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pgp_num","val":"11"}]': finished 2024-09-11T08:04:54.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[41371]: osdmap e166: 8 total, 8 up, 8 in 2024-09-11T08:04:54.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:54.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:54.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1182028243' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi102-80356-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:54.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:54.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[41371]: from='client.47842 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi102-80356-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:54.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3656025770' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:54.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[41371]: from='client.46733 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:54.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[29849]: pgmap v215: 300 pgs: 40 unknown, 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:04:54.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:54.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[29849]: from='client.24692 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:04:54.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[29849]: from='client.24692 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pgp_num","val":"11"}]': finished 2024-09-11T08:04:54.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[29849]: osdmap e166: 8 total, 8 up, 8 in 2024-09-11T08:04:54.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:54.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:54.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1182028243' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi102-80356-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:54.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:54.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[29849]: from='client.47842 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi102-80356-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:54.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3656025770' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:54.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:54 smithi102 ceph-mon[29849]: from='client.46733 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:54.684 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:04:54 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:04:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:04:54.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:54 smithi146 ceph-mon[33158]: pgmap v215: 300 pgs: 40 unknown, 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:04:54.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:54 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:04:54.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:54 smithi146 ceph-mon[33158]: from='client.24692 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-11T08:04:54.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:54 smithi146 ceph-mon[33158]: from='client.24692 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi102-80411-1","var":"pgp_num","val":"11"}]': finished 2024-09-11T08:04:54.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:54 smithi146 ceph-mon[33158]: osdmap e166: 8 total, 8 up, 8 in 2024-09-11T08:04:54.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:54 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:54.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:54 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:54.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:54 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1182028243' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi102-80356-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:54.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:54 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:54.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:54 smithi146 ceph-mon[33158]: from='client.47842 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi102-80356-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:54.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:54 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3656025770' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:54.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:54 smithi146 ceph-mon[33158]: from='client.46733 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:56.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:55 smithi102 ceph-mon[29849]: pgmap v218: 292 pgs: 64 unknown, 228 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:04:56.149 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:55 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:56.149 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:55 smithi102 ceph-mon[29849]: from='client.47842 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi102-80356-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:56.149 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:55 smithi102 ceph-mon[29849]: from='client.46733 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi102-80338-29"}]': finished 2024-09-11T08:04:56.149 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:55 smithi102 ceph-mon[29849]: osdmap e167: 8 total, 8 up, 8 in 2024-09-11T08:04:56.149 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:56.149 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3656025770' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:56.149 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:55 smithi102 ceph-mon[29849]: from='client.46733 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:56.149 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:04:56.149 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:55 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:04:56.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:55 smithi102 ceph-mon[41371]: pgmap v218: 292 pgs: 64 unknown, 228 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:04:56.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:55 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:56.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:55 smithi102 ceph-mon[41371]: from='client.47842 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi102-80356-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:56.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:55 smithi102 ceph-mon[41371]: from='client.46733 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi102-80338-29"}]': finished 2024-09-11T08:04:56.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:55 smithi102 ceph-mon[41371]: osdmap e167: 8 total, 8 up, 8 in 2024-09-11T08:04:56.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:56.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3656025770' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:56.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:55 smithi102 ceph-mon[41371]: from='client.46733 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:56.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:04:56.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:55 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:04:56.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:55 smithi146 ceph-mon[33158]: pgmap v218: 292 pgs: 64 unknown, 228 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:04:56.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:55 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:56.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:55 smithi146 ceph-mon[33158]: from='client.47842 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi102-80356-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:56.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:55 smithi146 ceph-mon[33158]: from='client.46733 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi102-80338-29"}]': finished 2024-09-11T08:04:56.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:55 smithi146 ceph-mon[33158]: osdmap e167: 8 total, 8 up, 8 in 2024-09-11T08:04:56.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:56.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3656025770' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:56.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:55 smithi146 ceph-mon[33158]: from='client.46733 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi102-80338-29"}]: dispatch 2024-09-11T08:04:56.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:04:56.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:55 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:04:57.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:56 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:57.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:56 smithi146 ceph-mon[33158]: from='client.46733 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi102-80338-29"}]': finished 2024-09-11T08:04:57.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:56 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:04:57.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:56 smithi146 ceph-mon[33158]: osdmap e168: 8 total, 8 up, 8 in 2024-09-11T08:04:57.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:56 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-17"}]: dispatch 2024-09-11T08:04:57.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:56 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-17"}]: dispatch 2024-09-11T08:04:57.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:56 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi102-80338-30"}]: dispatch 2024-09-11T08:04:57.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:56 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi102-80338-30"}]: dispatch 2024-09-11T08:04:57.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:56 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi102-80338-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:57.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:56 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi102-80411-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-11T08:04:57.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:56 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi102-80411-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-11T08:04:57.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:56 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[41371]: from='client.46733 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi102-80338-29"}]': finished 2024-09-11T08:04:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:04:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[41371]: osdmap e168: 8 total, 8 up, 8 in 2024-09-11T08:04:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-17"}]: dispatch 2024-09-11T08:04:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-17"}]: dispatch 2024-09-11T08:04:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi102-80338-30"}]: dispatch 2024-09-11T08:04:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi102-80338-30"}]: dispatch 2024-09-11T08:04:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi102-80338-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi102-80411-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-11T08:04:57.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi102-80411-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-11T08:04:57.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:57.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:57.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[29849]: from='client.46733 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi102-80338-29"}]': finished 2024-09-11T08:04:57.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:04:57.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[29849]: osdmap e168: 8 total, 8 up, 8 in 2024-09-11T08:04:57.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-17"}]: dispatch 2024-09-11T08:04:57.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-17"}]: dispatch 2024-09-11T08:04:57.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi102-80338-30"}]: dispatch 2024-09-11T08:04:57.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi102-80338-30"}]: dispatch 2024-09-11T08:04:57.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi102-80338-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:04:57.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi102-80411-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-11T08:04:57.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi102-80411-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-11T08:04:57.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:56 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:58.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:57 smithi146 ceph-mon[33158]: pgmap v221: 260 pgs: 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-11T08:04:58.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:57 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-17"}]': finished 2024-09-11T08:04:58.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:57 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi102-80338-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:58.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:57 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi102-80411-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-11T08:04:58.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:57 smithi146 ceph-mon[33158]: osdmap e169: 8 total, 8 up, 8 in 2024-09-11T08:04:58.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:57 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi102-80338-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi102-80338-30"}]: dispatch 2024-09-11T08:04:58.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:57 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1923181577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi102-80356-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:58.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:57 smithi146 ceph-mon[33158]: from='client.46751 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi102-80356-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:58.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:57 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-17", "mode": "writeback"}]: dispatch 2024-09-11T08:04:58.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:57 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-17", "mode": "writeback"}]: dispatch 2024-09-11T08:04:58.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:57 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:58.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:57 smithi102 ceph-mon[41371]: pgmap v221: 260 pgs: 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-11T08:04:58.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:57 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-17"}]': finished 2024-09-11T08:04:58.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:57 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi102-80338-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:58.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:57 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi102-80411-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-11T08:04:58.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:57 smithi102 ceph-mon[41371]: osdmap e169: 8 total, 8 up, 8 in 2024-09-11T08:04:58.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:57 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi102-80338-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi102-80338-30"}]: dispatch 2024-09-11T08:04:58.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:57 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1923181577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi102-80356-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:58.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:57 smithi102 ceph-mon[41371]: from='client.46751 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi102-80356-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:58.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:57 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-17", "mode": "writeback"}]: dispatch 2024-09-11T08:04:58.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:57 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-17", "mode": "writeback"}]: dispatch 2024-09-11T08:04:58.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:57 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:58.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:57 smithi102 ceph-mon[29849]: pgmap v221: 260 pgs: 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-11T08:04:58.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:57 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-17"}]': finished 2024-09-11T08:04:58.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:57 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi102-80338-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:04:58.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:57 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi102-80411-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-11T08:04:58.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:57 smithi102 ceph-mon[29849]: osdmap e169: 8 total, 8 up, 8 in 2024-09-11T08:04:58.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:57 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi102-80338-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi102-80338-30"}]: dispatch 2024-09-11T08:04:58.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:57 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1923181577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi102-80356-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:58.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:57 smithi102 ceph-mon[29849]: from='client.46751 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi102-80356-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:04:58.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:57 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-17", "mode": "writeback"}]: dispatch 2024-09-11T08:04:58.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:57 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-17", "mode": "writeback"}]: dispatch 2024-09-11T08:04:58.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:57 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/121365372' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-11T08:04:59.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:58 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:04:59.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:58 smithi146 ceph-mon[33158]: from='client.46751 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi102-80356-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:59.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:58 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-17", "mode": "writeback"}]': finished 2024-09-11T08:04:59.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:58 smithi146 ceph-mon[33158]: osdmap e170: 8 total, 8 up, 8 in 2024-09-11T08:04:59.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:58 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:04:59.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:58 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:04:59.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:58 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:04:59.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:58 smithi102 ceph-mon[41371]: from='client.46751 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi102-80356-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:59.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:58 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-17", "mode": "writeback"}]': finished 2024-09-11T08:04:59.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:58 smithi102 ceph-mon[41371]: osdmap e170: 8 total, 8 up, 8 in 2024-09-11T08:04:59.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:58 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:04:59.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:58 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:04:59.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:58 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:04:59.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:58 smithi102 ceph-mon[29849]: from='client.46751 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi102-80356-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:04:59.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:58 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-17", "mode": "writeback"}]': finished 2024-09-11T08:04:59.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:58 smithi102 ceph-mon[29849]: osdmap e170: 8 total, 8 up, 8 in 2024-09-11T08:04:59.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:58 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:04:59.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:58 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:04:59.904 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:04:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:04:59.906 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:59 smithi146 ceph-mon[33158]: pgmap v224: 292 pgs: 32 unknown, 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-11T08:04:59.906 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:59 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi102-80338-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi102-80338-30"}]': finished 2024-09-11T08:04:59.906 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:59 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:04:59.906 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:59 smithi146 ceph-mon[33158]: osdmap e171: 8 total, 8 up, 8 in 2024-09-11T08:04:59.906 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:59 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-17"}]: dispatch 2024-09-11T08:04:59.906 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:59 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-17"}]: dispatch 2024-09-11T08:04:59.906 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:59 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:04:59.906 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:04:59 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:05:00.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:04:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:04:59] "GET /metrics HTTP/1.1" 200 233186 "" "Prometheus/2.43.0" 2024-09-11T08:05:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:59 smithi102 ceph-mon[29849]: pgmap v224: 292 pgs: 32 unknown, 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-11T08:05:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:59 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi102-80338-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi102-80338-30"}]': finished 2024-09-11T08:05:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:59 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:05:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:59 smithi102 ceph-mon[29849]: osdmap e171: 8 total, 8 up, 8 in 2024-09-11T08:05:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:59 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-17"}]: dispatch 2024-09-11T08:05:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:59 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-17"}]: dispatch 2024-09-11T08:05:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:59 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:05:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:04:59 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:05:00.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:59 smithi102 ceph-mon[41371]: pgmap v224: 292 pgs: 32 unknown, 260 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-11T08:05:00.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:59 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi102-80338-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi102-80338-30"}]': finished 2024-09-11T08:05:00.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:59 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:05:00.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:59 smithi102 ceph-mon[41371]: osdmap e171: 8 total, 8 up, 8 in 2024-09-11T08:05:00.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:59 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-17"}]: dispatch 2024-09-11T08:05:00.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:59 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-17"}]: dispatch 2024-09-11T08:05:00.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:59 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:05:00.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:04:59 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:05:01.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:00 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:05:01.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:00 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:05:01.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:00 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-17"}]': finished 2024-09-11T08:05:01.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:00 smithi146 ceph-mon[33158]: osdmap e172: 8 total, 8 up, 8 in 2024-09-11T08:05:01.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:00 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4181877629' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi102-80356-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:01.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:00 smithi146 ceph-mon[33158]: from='client.46760 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi102-80356-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:01.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:00 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:05:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:00 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:05:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:00 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-17"}]': finished 2024-09-11T08:05:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:00 smithi102 ceph-mon[29849]: osdmap e172: 8 total, 8 up, 8 in 2024-09-11T08:05:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:00 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4181877629' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi102-80356-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:00 smithi102 ceph-mon[29849]: from='client.46760 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi102-80356-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:01.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:00 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:05:01.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:00 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:05:01.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:00 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-17"}]': finished 2024-09-11T08:05:01.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:00 smithi102 ceph-mon[41371]: osdmap e172: 8 total, 8 up, 8 in 2024-09-11T08:05:01.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:00 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4181877629' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi102-80356-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:01.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:00 smithi102 ceph-mon[41371]: from='client.46760 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi102-80356-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:02.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:01 smithi146 ceph-mon[33158]: pgmap v227: 299 pgs: 32 creating+peering, 267 active+clean; 595 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.5 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-09-11T08:05:02.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:01 smithi146 ceph-mon[33158]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:02.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:01 smithi146 ceph-mon[33158]: from='client.46760 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi102-80356-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:02.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:01 smithi146 ceph-mon[33158]: osdmap e173: 8 total, 8 up, 8 in 2024-09-11T08:05:02.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:01 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi102-80338-30"}]: dispatch 2024-09-11T08:05:02.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:01 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2428574926' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:02.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:01 smithi146 ceph-mon[33158]: from='client.46766 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:02.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:01 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2428574926' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:02.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:01 smithi146 ceph-mon[33158]: from='client.46766 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:02.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:01 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2428574926' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi102-80411-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:02.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:01 smithi146 ceph-mon[33158]: from='client.46766 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi102-80411-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:01 smithi102 ceph-mon[29849]: pgmap v227: 299 pgs: 32 creating+peering, 267 active+clean; 595 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.5 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-09-11T08:05:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:01 smithi102 ceph-mon[29849]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:01 smithi102 ceph-mon[29849]: from='client.46760 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi102-80356-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:01 smithi102 ceph-mon[29849]: osdmap e173: 8 total, 8 up, 8 in 2024-09-11T08:05:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:01 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi102-80338-30"}]: dispatch 2024-09-11T08:05:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:01 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2428574926' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:01 smithi102 ceph-mon[29849]: from='client.46766 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:01 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2428574926' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:01 smithi102 ceph-mon[29849]: from='client.46766 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:01 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2428574926' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi102-80411-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:02.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:01 smithi102 ceph-mon[29849]: from='client.46766 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi102-80411-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:02.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:01 smithi102 ceph-mon[41371]: pgmap v227: 299 pgs: 32 creating+peering, 267 active+clean; 595 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.5 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-09-11T08:05:02.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:01 smithi102 ceph-mon[41371]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:02.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:01 smithi102 ceph-mon[41371]: from='client.46760 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi102-80356-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:02.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:01 smithi102 ceph-mon[41371]: osdmap e173: 8 total, 8 up, 8 in 2024-09-11T08:05:02.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:01 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi102-80338-30"}]: dispatch 2024-09-11T08:05:02.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:01 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2428574926' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:02.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:01 smithi102 ceph-mon[41371]: from='client.46766 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:02.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:01 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2428574926' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:02.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:01 smithi102 ceph-mon[41371]: from='client.46766 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:02.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:01 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2428574926' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi102-80411-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:02.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:01 smithi102 ceph-mon[41371]: from='client.46766 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi102-80411-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:03.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:02 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi102-80338-30"}]': finished 2024-09-11T08:05:03.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:02 smithi146 ceph-mon[33158]: from='client.46766 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi102-80411-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:03.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:02 smithi146 ceph-mon[33158]: osdmap e174: 8 total, 8 up, 8 in 2024-09-11T08:05:03.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:02 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi102-80338-30"}]: dispatch 2024-09-11T08:05:03.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:02 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2428574926' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi102-80411-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:03.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:02 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:03.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:02 smithi146 ceph-mon[33158]: from='client.46766 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi102-80411-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:03.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:02 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:03.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:02 smithi146 ceph-mon[33158]: pgmap v230: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:05:03.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:02 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi102-80338-30"}]': finished 2024-09-11T08:05:03.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:02 smithi102 ceph-mon[41371]: from='client.46766 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi102-80411-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:03.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:02 smithi102 ceph-mon[41371]: osdmap e174: 8 total, 8 up, 8 in 2024-09-11T08:05:03.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:02 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi102-80338-30"}]: dispatch 2024-09-11T08:05:03.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:02 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2428574926' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi102-80411-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:03.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:02 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:03.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:02 smithi102 ceph-mon[41371]: from='client.46766 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi102-80411-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:03.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:02 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:03.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:02 smithi102 ceph-mon[41371]: pgmap v230: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:05:03.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:02 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi102-80338-30"}]': finished 2024-09-11T08:05:03.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:02 smithi102 ceph-mon[29849]: from='client.46766 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi102-80411-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:03.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:02 smithi102 ceph-mon[29849]: osdmap e174: 8 total, 8 up, 8 in 2024-09-11T08:05:03.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:02 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi102-80338-30"}]: dispatch 2024-09-11T08:05:03.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:02 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2428574926' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi102-80411-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:03.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:02 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:03.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:02 smithi102 ceph-mon[29849]: from='client.46766 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi102-80411-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:03.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:02 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:03.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:02 smithi102 ceph-mon[29849]: pgmap v230: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:05:04.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:03 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi102-80338-30"}]': finished 2024-09-11T08:05:04.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:03 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:04.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:03 smithi146 ceph-mon[33158]: osdmap e175: 8 total, 8 up, 8 in 2024-09-11T08:05:04.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:03 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:04.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:03 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/937955100' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi102-80356-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:04.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:03 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:04.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:03 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2855483508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:04.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:03 smithi146 ceph-mon[33158]: from='client.46778 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:04.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:03 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2855483508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:04.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:03 smithi146 ceph-mon[33158]: from='client.46778 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:04.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:03 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2855483508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi102-80338-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:04.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:03 smithi146 ceph-mon[33158]: from='client.46778 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi102-80338-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:04.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi102-80338-30"}]': finished 2024-09-11T08:05:04.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:04.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[41371]: osdmap e175: 8 total, 8 up, 8 in 2024-09-11T08:05:04.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:04.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/937955100' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi102-80356-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:04.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:04.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2855483508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:04.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[41371]: from='client.46778 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:04.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2855483508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:04.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[41371]: from='client.46778 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:04.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2855483508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi102-80338-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:04.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[41371]: from='client.46778 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi102-80338-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:04.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2771076421' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi102-80338-30"}]': finished 2024-09-11T08:05:04.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:04.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[29849]: osdmap e175: 8 total, 8 up, 8 in 2024-09-11T08:05:04.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:04.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/937955100' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi102-80356-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:04.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:04.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2855483508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:04.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[29849]: from='client.46778 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:04.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2855483508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:04.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[29849]: from='client.46778 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:04.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2855483508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi102-80338-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:04.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:03 smithi102 ceph-mon[29849]: from='client.46778 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi102-80338-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:04.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:05:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:05:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:05:05.173 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio: cursor()=9:bd63b0f1:::8:head expected=9:bd63b0f1:::8:head 2024-09-11T08:05:05.173 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:bd63b0f1:::8:head -> 8 2024-09-11T08:05:05.173 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : entry=8 expected=8 2024-09-11T08:05:05.173 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:b29083e3:::5:head 2024-09-11T08:05:05.173 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : cursor()=9:b29083e3:::5:head expected=9:b29083e3:::5:head 2024-09-11T08:05:05.173 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:b29083e3:::5:head -> 5 2024-09-11T08:05:05.173 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : entry=5 expected=5 2024-09-11T08:05:05.173 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:863748b0:::15:head 2024-09-11T08:05:05.173 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : cursor()=9:863748b0:::15:head expected=9:863748b0:::15:head 2024-09-11T08:05:05.173 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:863748b0:::15:head -> 15 2024-09-11T08:05:05.174 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : entry=15 expected=15 2024-09-11T08:05:05.174 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:6cac518f:::0:head 2024-09-11T08:05:05.174 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : cursor()=9:6cac518f:::0:head expected=9:6cac518f:::0:head 2024-09-11T08:05:05.174 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:6cac518f:::0:head -> 0 2024-09-11T08:05:05.174 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : entry=0 expected=0 2024-09-11T08:05:05.174 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:62a1935d:::14:head 2024-09-11T08:05:05.174 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : cursor()=9:62a1935d:::14:head expected=9:62a1935d:::14:head 2024-09-11T08:05:05.174 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:62a1935d:::14:head -> 14 2024-09-11T08:05:05.174 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : entry=14 expected=14 2024-09-11T08:05:05.174 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:5c6b0b28:::7:head 2024-09-11T08:05:05.174 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : cursor()=9:5c6b0b28:::7:head expected=9:5c6b0b28:::7:head 2024-09-11T08:05:05.174 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:5c6b0b28:::7:head -> 7 2024-09-11T08:05:05.174 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : entry=7 expected=7 2024-09-11T08:05:05.175 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:566253c9:::13:head 2024-09-11T08:05:05.175 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : cursor()=9:566253c9:::13:head expected=9:566253c9:::13:head 2024-09-11T08:05:05.175 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:566253c9:::13:head -> 13 2024-09-11T08:05:05.175 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : entry=13 expected=13 2024-09-11T08:05:05.175 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : seek to 9:52ea6a34:::10:head 2024-09-11T08:05:05.175 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : cursor()=9:52ea6a34:::10:head expected=9:52ea6a34:::10:head 2024-09-11T08:05:05.175 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: > 9:52ea6a34:::10:head -> 10 2024-09-11T08:05:05.175 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: : entry=10 expected=10 2024-09-11T08:05:05.175 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (218 ms) 2024-09-11T08:05:05.175 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-09-11T08:05:05.175 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (96263 ms) 2024-09-11T08:05:05.175 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-09-11T08:05:05.175 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: split 0/5 -> MIN 9:33333333::::head 2024-09-11T08:05:05.175 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: split 1/5 -> 9:33333333::::head 9:66666666::::head 2024-09-11T08:05:05.176 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: split 2/5 -> 9:66666666::::head 9:99999999::::head 2024-09-11T08:05:05.176 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: split 3/5 -> 9:99999999::::head 9:cccccccc::::head 2024-09-11T08:05:05.176 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: split 4/5 -> 9:cccccccc::::head MAX 2024-09-11T08:05:05.176 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (8022 ms) 2024-09-11T08:05:05.176 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [----------] 7 tests from LibRadosList (105009 ms total) 2024-09-11T08:05:05.176 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 2024-09-11T08:05:05.176 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-09-11T08:05:05.176 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-09-11T08:05:05.176 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1185 ms) 2024-09-11T08:05:05.176 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-09-11T08:05:05.176 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: myset foo1,foo2,foo3 2024-09-11T08:05:05.176 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: foo1 2024-09-11T08:05:05.177 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: foo2 2024-09-11T08:05:05.177 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: foo3 2024-09-11T08:05:05.177 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: myset foo1,foo4,foo5 2024-09-11T08:05:05.177 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: foo4 2024-09-11T08:05:05.177 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: foo5 2024-09-11T08:05:05.177 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: foo1 2024-09-11T08:05:05.177 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: myset foo6,foo7 2024-09-11T08:05:05.177 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: foo7 2024-09-11T08:05:05.177 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: foo6 2024-09-11T08:05:05.177 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-09-11T08:05:05.177 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: ns1:foo4 2024-09-11T08:05:05.177 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: ns1:foo5 2024-09-11T08:05:05.177 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: ns2:foo7 2024-09-11T08:05:05.177 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: ns2:foo6 2024-09-11T08:05:05.177 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: ns1:foo1 2024-09-11T08:05:05.178 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: :foo1 2024-09-11T08:05:05.178 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: :foo2 2024-09-11T08:05:05.178 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: :foo3 2024-09-11T08:05:05.178 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (26 ms) 2024-09-11T08:05:05.178 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2024-09-11T08:05:05.178 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 1 0 2024-09-11T08:05:05.178 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 10 0 2024-09-11T08:05:05.178 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 13 0 2024-09-11T08:05:05.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:04 smithi102 ceph-mon[29849]: from='client.46766 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi102-80411-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi102-80411-2"}]': finished 2024-09-11T08:05:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:04 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/937955100' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi102-80356-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:04 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:05:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:04 smithi102 ceph-mon[29849]: from='client.46778 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi102-80338-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:04 smithi102 ceph-mon[29849]: osdmap e176: 8 total, 8 up, 8 in 2024-09-11T08:05:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:04 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-19"}]: dispatch 2024-09-11T08:05:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:04 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2855483508' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi102-80338-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:04 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-19"}]: dispatch 2024-09-11T08:05:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:04 smithi102 ceph-mon[29849]: from='client.46778 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi102-80338-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:04 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/937955100' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi102-80356-33", "var": "eio", "val": "true"}]: dispatch 2024-09-11T08:05:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:04 smithi102 ceph-mon[29849]: pgmap v233: 268 pgs: 72 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:05:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:04 smithi102 ceph-mon[41371]: from='client.46766 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi102-80411-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi102-80411-2"}]': finished 2024-09-11T08:05:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:04 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/937955100' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi102-80356-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:04 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:05:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:04 smithi102 ceph-mon[41371]: from='client.46778 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi102-80338-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:04 smithi102 ceph-mon[41371]: osdmap e176: 8 total, 8 up, 8 in 2024-09-11T08:05:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:04 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-19"}]: dispatch 2024-09-11T08:05:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:04 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2855483508' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi102-80338-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:04 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-19"}]: dispatch 2024-09-11T08:05:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:04 smithi102 ceph-mon[41371]: from='client.46778 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi102-80338-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:04 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/937955100' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi102-80356-33", "var": "eio", "val": "true"}]: dispatch 2024-09-11T08:05:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:04 smithi102 ceph-mon[41371]: pgmap v233: 268 pgs: 72 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:05:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:04 smithi146 ceph-mon[33158]: from='client.46766 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi102-80411-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi102-80411-2"}]': finished 2024-09-11T08:05:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:04 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/937955100' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi102-80356-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:04 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:05:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:04 smithi146 ceph-mon[33158]: from='client.46778 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi102-80338-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:04 smithi146 ceph-mon[33158]: osdmap e176: 8 total, 8 up, 8 in 2024-09-11T08:05:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:04 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-19"}]: dispatch 2024-09-11T08:05:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:04 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2855483508' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi102-80338-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:04 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-19"}]: dispatch 2024-09-11T08:05:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:04 smithi146 ceph-mon[33158]: from='client.46778 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi102-80338-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:04 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/937955100' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi102-80356-33", "var": "eio", "val": "true"}]: dispatch 2024-09-11T08:05:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:04 smithi146 ceph-mon[33158]: pgmap v233: 268 pgs: 72 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:05:06.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:05 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-19"}]': finished 2024-09-11T08:05:06.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:05 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-19", "mode": "writeback"}]: dispatch 2024-09-11T08:05:06.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:05 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/937955100' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi102-80356-33", "var": "eio", "val": "true"}]': finished 2024-09-11T08:05:06.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:05 smithi102 ceph-mon[29849]: osdmap e177: 8 total, 8 up, 8 in 2024-09-11T08:05:06.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:05 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-19", "mode": "writeback"}]: dispatch 2024-09-11T08:05:06.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:05 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-19"}]': finished 2024-09-11T08:05:06.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:05 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-19", "mode": "writeback"}]: dispatch 2024-09-11T08:05:06.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:05 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/937955100' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi102-80356-33", "var": "eio", "val": "true"}]': finished 2024-09-11T08:05:06.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:05 smithi102 ceph-mon[41371]: osdmap e177: 8 total, 8 up, 8 in 2024-09-11T08:05:06.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:05 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-19", "mode": "writeback"}]: dispatch 2024-09-11T08:05:06.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:05 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-19"}]': finished 2024-09-11T08:05:06.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:05 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-19", "mode": "writeback"}]: dispatch 2024-09-11T08:05:06.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:05 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/937955100' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi102-80356-33", "var": "eio", "val": "true"}]': finished 2024-09-11T08:05:06.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:05 smithi146 ceph-mon[33158]: osdmap e177: 8 total, 8 up, 8 in 2024-09-11T08:05:06.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:05 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-19", "mode": "writeback"}]: dispatch 2024-09-11T08:05:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:06 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:05:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:06 smithi102 ceph-mon[29849]: from='client.46778 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi102-80338-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi102-80338-31"}]': finished 2024-09-11T08:05:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:06 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-19", "mode": "writeback"}]': finished 2024-09-11T08:05:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:06 smithi102 ceph-mon[29849]: osdmap e178: 8 total, 8 up, 8 in 2024-09-11T08:05:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:06 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2428574926' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:06 smithi102 ceph-mon[29849]: from='client.46766 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:06 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:07.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:06 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:07.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:06 smithi102 ceph-mon[29849]: pgmap v236: 236 pgs: 8 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:05:07.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:06 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:07.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:06 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:05:07.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:06 smithi102 ceph-mon[41371]: from='client.46778 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi102-80338-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi102-80338-31"}]': finished 2024-09-11T08:05:07.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:06 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-19", "mode": "writeback"}]': finished 2024-09-11T08:05:07.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:06 smithi102 ceph-mon[41371]: osdmap e178: 8 total, 8 up, 8 in 2024-09-11T08:05:07.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:06 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2428574926' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:07.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:06 smithi102 ceph-mon[41371]: from='client.46766 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:07.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:06 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:07.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:06 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:07.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:06 smithi102 ceph-mon[41371]: pgmap v236: 236 pgs: 8 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:05:07.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:06 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:06 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:05:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:06 smithi146 ceph-mon[33158]: from='client.46778 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi102-80338-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi102-80338-31"}]': finished 2024-09-11T08:05:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:06 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-19", "mode": "writeback"}]': finished 2024-09-11T08:05:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:06 smithi146 ceph-mon[33158]: osdmap e178: 8 total, 8 up, 8 in 2024-09-11T08:05:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:06 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2428574926' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:06 smithi146 ceph-mon[33158]: from='client.46766 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:06 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:06 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:07.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:06 smithi146 ceph-mon[33158]: pgmap v236: 236 pgs: 8 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:05:07.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:06 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:08.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:07 smithi102 ceph-mon[29849]: from='client.46766 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi102-80411-2"}]': finished 2024-09-11T08:05:08.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:07 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:05:08.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:07 smithi102 ceph-mon[29849]: osdmap e179: 8 total, 8 up, 8 in 2024-09-11T08:05:08.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2428574926' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:08.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-19"}]: dispatch 2024-09-11T08:05:08.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:07 smithi102 ceph-mon[29849]: from='client.46766 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:08.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3975862095' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi102-80356-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:08.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:07 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-19"}]: dispatch 2024-09-11T08:05:08.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:07 smithi102 ceph-mon[41371]: from='client.46766 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi102-80411-2"}]': finished 2024-09-11T08:05:08.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:07 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:05:08.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:07 smithi102 ceph-mon[41371]: osdmap e179: 8 total, 8 up, 8 in 2024-09-11T08:05:08.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2428574926' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:08.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-19"}]: dispatch 2024-09-11T08:05:08.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:07 smithi102 ceph-mon[41371]: from='client.46766 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:08.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3975862095' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi102-80356-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:08.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:07 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-19"}]: dispatch 2024-09-11T08:05:08.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:07 smithi146 ceph-mon[33158]: from='client.46766 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi102-80411-2"}]': finished 2024-09-11T08:05:08.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:07 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:05:08.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:07 smithi146 ceph-mon[33158]: osdmap e179: 8 total, 8 up, 8 in 2024-09-11T08:05:08.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2428574926' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:08.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-19"}]: dispatch 2024-09-11T08:05:08.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:07 smithi146 ceph-mon[33158]: from='client.46766 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi102-80411-2"}]: dispatch 2024-09-11T08:05:08.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3975862095' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi102-80356-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:08.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:07 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-19"}]: dispatch 2024-09-11T08:05:09.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:08 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:05:09.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:08 smithi102 ceph-mon[41371]: from='client.46766 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi102-80411-2"}]': finished 2024-09-11T08:05:09.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:08 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3975862095' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi102-80356-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:09.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:08 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-19"}]': finished 2024-09-11T08:05:09.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:08 smithi102 ceph-mon[41371]: osdmap e180: 8 total, 8 up, 8 in 2024-09-11T08:05:09.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:08 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2855483508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:09.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:08 smithi102 ceph-mon[41371]: from='client.46778 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:09.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:08 smithi102 ceph-mon[41371]: pgmap v239: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:05:09.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:08 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:05:09.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:08 smithi102 ceph-mon[29849]: from='client.46766 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi102-80411-2"}]': finished 2024-09-11T08:05:09.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:08 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3975862095' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi102-80356-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:09.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:08 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-19"}]': finished 2024-09-11T08:05:09.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:08 smithi102 ceph-mon[29849]: osdmap e180: 8 total, 8 up, 8 in 2024-09-11T08:05:09.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:08 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2855483508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:09.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:08 smithi102 ceph-mon[29849]: from='client.46778 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:09.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:08 smithi102 ceph-mon[29849]: pgmap v239: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:05:09.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:08 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:05:09.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:08 smithi146 ceph-mon[33158]: from='client.46766 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi102-80411-2"}]': finished 2024-09-11T08:05:09.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:08 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3975862095' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi102-80356-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:09.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:08 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-19"}]': finished 2024-09-11T08:05:09.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:08 smithi146 ceph-mon[33158]: osdmap e180: 8 total, 8 up, 8 in 2024-09-11T08:05:09.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:08 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2855483508' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:09.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:08 smithi146 ceph-mon[33158]: from='client.46778 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:09.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:08 smithi146 ceph-mon[33158]: pgmap v239: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:05:09.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:05:09 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:05:10.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:05:09 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:05:09] "GET /metrics HTTP/1.1" 200 233186 "" "Prometheus/2.43.0" 2024-09-11T08:05:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:09 smithi102 ceph-mon[41371]: from='client.46778 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi102-80338-31"}]': finished 2024-09-11T08:05:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:09 smithi102 ceph-mon[41371]: osdmap e181: 8 total, 8 up, 8 in 2024-09-11T08:05:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:09 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2855483508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:09 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3316080617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80411-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:09 smithi102 ceph-mon[41371]: from='client.46778 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:09 smithi102 ceph-mon[41371]: from='client.47890 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80411-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:09 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:05:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:09 smithi102 ceph-mon[29849]: from='client.46778 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi102-80338-31"}]': finished 2024-09-11T08:05:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:09 smithi102 ceph-mon[29849]: osdmap e181: 8 total, 8 up, 8 in 2024-09-11T08:05:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:09 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2855483508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:09 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3316080617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80411-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:09 smithi102 ceph-mon[29849]: from='client.46778 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:09 smithi102 ceph-mon[29849]: from='client.47890 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80411-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:09 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:05:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:09 smithi146 ceph-mon[33158]: from='client.46778 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi102-80338-31"}]': finished 2024-09-11T08:05:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:09 smithi146 ceph-mon[33158]: osdmap e181: 8 total, 8 up, 8 in 2024-09-11T08:05:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:09 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2855483508' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:09 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3316080617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80411-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:09 smithi146 ceph-mon[33158]: from='client.46778 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi102-80338-31"}]: dispatch 2024-09-11T08:05:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:09 smithi146 ceph-mon[33158]: from='client.47890 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80411-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:10.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:09 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:05:11.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 7 0 2024-09-11T08:05:11.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 14 0 2024-09-11T08:05:11.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 0 0 2024-09-11T08:05:11.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 15 0 2024-09-11T08:05:11.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 11 0 2024-09-11T08:05:11.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 5 0 2024-09-11T08:05:11.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 8 0 2024-09-11T08:05:11.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 6 0 2024-09-11T08:05:11.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 3 0 2024-09-11T08:05:11.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 4 0 2024-09-11T08:05:11.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 12 0 2024-09-11T08:05:11.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 9 0 2024-09-11T08:05:11.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 2 0 2024-09-11T08:05:11.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-09-11T08:05:11.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (39 ms) 2024-09-11T08:05:11.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [----------] 3 tests from LibRadosListEC (1250 ms total) 2024-09-11T08:05:11.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 2024-09-11T08:05:11.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-09-11T08:05:11.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-09-11T08:05:11.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3034 ms) 2024-09-11T08:05:11.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [----------] 1 test from LibRadosListNP (3034 ms total) 2024-09-11T08:05:11.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: 2024-09-11T08:05:11.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [----------] Global test environment tear-down 2024-09-11T08:05:11.016 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [==========] 11 tests from 3 test suites ran. (122844 ms total) 2024-09-11T08:05:11.017 INFO:tasks.workunit.client.0.smithi102.stdout: api_list: [ PASSED ] 11 tests. 2024-09-11T08:05:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[41371]: from='client.46778 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi102-80338-31"}]': finished 2024-09-11T08:05:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[41371]: from='client.47890 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80411-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[41371]: osdmap e182: 8 total, 8 up, 8 in 2024-09-11T08:05:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1388104259' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi102-80356-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[41371]: from='client.47896 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi102-80356-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:11.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3316080617' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi102-80411-3","pool2":"test-rados-api-smithi102-80411-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-11T08:05:11.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[41371]: from='client.47890 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi102-80411-3","pool2":"test-rados-api-smithi102-80411-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-11T08:05:11.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3295115296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:11.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[41371]: from='client.46802 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:11.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3295115296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:11.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[41371]: from='client.46802 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:11.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3295115296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi102-80338-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:11.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[41371]: from='client.46802 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi102-80338-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:11.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[41371]: pgmap v242: 292 pgs: 64 creating+peering, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:05:11.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[29849]: from='client.46778 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi102-80338-31"}]': finished 2024-09-11T08:05:11.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[29849]: from='client.47890 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80411-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:11.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[29849]: osdmap e182: 8 total, 8 up, 8 in 2024-09-11T08:05:11.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:11.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1388104259' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi102-80356-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:11.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:11.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[29849]: from='client.47896 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi102-80356-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:11.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3316080617' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi102-80411-3","pool2":"test-rados-api-smithi102-80411-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-11T08:05:11.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[29849]: from='client.47890 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi102-80411-3","pool2":"test-rados-api-smithi102-80411-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-11T08:05:11.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3295115296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:11.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[29849]: from='client.46802 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:11.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3295115296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:11.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[29849]: from='client.46802 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:11.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3295115296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi102-80338-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:11.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[29849]: from='client.46802 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi102-80338-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:11.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:10 smithi102 ceph-mon[29849]: pgmap v242: 292 pgs: 64 creating+peering, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:05:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:10 smithi146 ceph-mon[33158]: from='client.46778 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi102-80338-31"}]': finished 2024-09-11T08:05:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:10 smithi146 ceph-mon[33158]: from='client.47890 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80411-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:10 smithi146 ceph-mon[33158]: osdmap e182: 8 total, 8 up, 8 in 2024-09-11T08:05:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1388104259' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi102-80356-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:10 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:10 smithi146 ceph-mon[33158]: from='client.47896 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi102-80356-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3316080617' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi102-80411-3","pool2":"test-rados-api-smithi102-80411-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-11T08:05:11.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:10 smithi146 ceph-mon[33158]: from='client.47890 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi102-80411-3","pool2":"test-rados-api-smithi102-80411-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-11T08:05:11.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3295115296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:11.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:10 smithi146 ceph-mon[33158]: from='client.46802 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:11.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3295115296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:11.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:10 smithi146 ceph-mon[33158]: from='client.46802 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:11.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3295115296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi102-80338-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:11.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:10 smithi146 ceph-mon[33158]: from='client.46802 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi102-80338-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:11.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:10 smithi146 ceph-mon[33158]: pgmap v242: 292 pgs: 64 creating+peering, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:05:12.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:12 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:12.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:12 smithi102 ceph-mon[41371]: from='client.47896 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi102-80356-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:12.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:12 smithi102 ceph-mon[41371]: from='client.47890 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi102-80411-3","pool2":"test-rados-api-smithi102-80411-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-11T08:05:12.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:12 smithi102 ceph-mon[41371]: from='client.46802 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi102-80338-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:12.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:12 smithi102 ceph-mon[41371]: osdmap e183: 8 total, 8 up, 8 in 2024-09-11T08:05:12.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3295115296' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi102-80338-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:12.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:12 smithi102 ceph-mon[41371]: from='client.46802 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi102-80338-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:12.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:12.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:12 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:12.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:12 smithi102 ceph-mon[41371]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:12.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:12 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:12.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:12 smithi102 ceph-mon[29849]: from='client.47896 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi102-80356-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:12.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:12 smithi102 ceph-mon[29849]: from='client.47890 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi102-80411-3","pool2":"test-rados-api-smithi102-80411-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-11T08:05:12.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:12 smithi102 ceph-mon[29849]: from='client.46802 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi102-80338-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:12.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:12 smithi102 ceph-mon[29849]: osdmap e183: 8 total, 8 up, 8 in 2024-09-11T08:05:12.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3295115296' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi102-80338-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:12.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:12 smithi102 ceph-mon[29849]: from='client.46802 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi102-80338-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:12.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:12.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:12 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:12.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:12 smithi102 ceph-mon[29849]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:12.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:12 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:12.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:12 smithi146 ceph-mon[33158]: from='client.47896 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi102-80356-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:12.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:12 smithi146 ceph-mon[33158]: from='client.47890 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi102-80411-3","pool2":"test-rados-api-smithi102-80411-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-11T08:05:12.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:12 smithi146 ceph-mon[33158]: from='client.46802 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi102-80338-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:12.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:12 smithi146 ceph-mon[33158]: osdmap e183: 8 total, 8 up, 8 in 2024-09-11T08:05:12.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3295115296' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi102-80338-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:12.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:12 smithi146 ceph-mon[33158]: from='client.46802 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi102-80338-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:12.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:12.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:12 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:12.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:12 smithi146 ceph-mon[33158]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:13.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:13 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:05:13.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:13 smithi102 ceph-mon[41371]: osdmap e184: 8 total, 8 up, 8 in 2024-09-11T08:05:13.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-21"}]: dispatch 2024-09-11T08:05:13.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:13 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-21"}]: dispatch 2024-09-11T08:05:13.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:13 smithi102 ceph-mon[41371]: pgmap v245: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:05:13.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:13 smithi102 ceph-mon[41371]: from='client.46802 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi102-80338-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi102-80338-32"}]': finished 2024-09-11T08:05:13.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:13 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-21"}]': finished 2024-09-11T08:05:13.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:13 smithi102 ceph-mon[41371]: osdmap e185: 8 total, 8 up, 8 in 2024-09-11T08:05:13.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-21", "mode": "writeback"}]: dispatch 2024-09-11T08:05:13.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4282943223' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi102-80356-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:13.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:13 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-21", "mode": "writeback"}]: dispatch 2024-09-11T08:05:13.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:13 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:05:13.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:13 smithi102 ceph-mon[29849]: osdmap e184: 8 total, 8 up, 8 in 2024-09-11T08:05:13.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-21"}]: dispatch 2024-09-11T08:05:13.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:13 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-21"}]: dispatch 2024-09-11T08:05:13.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:13 smithi102 ceph-mon[29849]: pgmap v245: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:05:13.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:13 smithi102 ceph-mon[29849]: from='client.46802 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi102-80338-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi102-80338-32"}]': finished 2024-09-11T08:05:13.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:13 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-21"}]': finished 2024-09-11T08:05:13.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:13 smithi102 ceph-mon[29849]: osdmap e185: 8 total, 8 up, 8 in 2024-09-11T08:05:13.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-21", "mode": "writeback"}]: dispatch 2024-09-11T08:05:13.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4282943223' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi102-80356-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:13.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:13 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-21", "mode": "writeback"}]: dispatch 2024-09-11T08:05:13.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:13 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:05:13.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:13 smithi146 ceph-mon[33158]: osdmap e184: 8 total, 8 up, 8 in 2024-09-11T08:05:13.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-21"}]: dispatch 2024-09-11T08:05:13.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:13 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-21"}]: dispatch 2024-09-11T08:05:13.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:13 smithi146 ceph-mon[33158]: pgmap v245: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:05:13.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:13 smithi146 ceph-mon[33158]: from='client.46802 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi102-80338-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi102-80338-32"}]': finished 2024-09-11T08:05:13.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:13 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-21"}]': finished 2024-09-11T08:05:13.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:13 smithi146 ceph-mon[33158]: osdmap e185: 8 total, 8 up, 8 in 2024-09-11T08:05:13.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-21", "mode": "writeback"}]: dispatch 2024-09-11T08:05:13.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4282943223' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi102-80356-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:13.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:13 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-21", "mode": "writeback"}]: dispatch 2024-09-11T08:05:14.588 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:14 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:05:14.588 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4282943223' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi102-80356-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:14.588 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:14 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-21", "mode": "writeback"}]': finished 2024-09-11T08:05:14.588 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:14 smithi146 ceph-mon[33158]: osdmap e186: 8 total, 8 up, 8 in 2024-09-11T08:05:14.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:14 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:05:14.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4282943223' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi102-80356-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:14.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:14 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-21", "mode": "writeback"}]': finished 2024-09-11T08:05:14.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:14 smithi102 ceph-mon[41371]: osdmap e186: 8 total, 8 up, 8 in 2024-09-11T08:05:14.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:14 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:05:14.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4282943223' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi102-80356-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:14.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:14 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-21", "mode": "writeback"}]': finished 2024-09-11T08:05:14.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:14 smithi102 ceph-mon[29849]: osdmap e186: 8 total, 8 up, 8 in 2024-09-11T08:05:14.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:05:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:05:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:05:15.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:15 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:15.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:15 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:15.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:05:15.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:15 smithi102 ceph-mon[29849]: pgmap v248: 268 pgs: 40 unknown, 32 creating+peering, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:05:15.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:15 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:05:15.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:15 smithi102 ceph-mon[29849]: osdmap e187: 8 total, 8 up, 8 in 2024-09-11T08:05:15.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:15 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1488050306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi102-80356-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:15.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:15 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-21"}]: dispatch 2024-09-11T08:05:15.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:15 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-21"}]: dispatch 2024-09-11T08:05:15.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:15 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3295115296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:15.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:15 smithi102 ceph-mon[29849]: from='client.46802 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:15.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:15 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:15.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:15 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:15.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:05:15.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:15 smithi102 ceph-mon[41371]: pgmap v248: 268 pgs: 40 unknown, 32 creating+peering, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:05:15.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:15 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:05:15.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:15 smithi102 ceph-mon[41371]: osdmap e187: 8 total, 8 up, 8 in 2024-09-11T08:05:15.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:15 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1488050306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi102-80356-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:15.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:15 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-21"}]: dispatch 2024-09-11T08:05:15.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:15 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-21"}]: dispatch 2024-09-11T08:05:15.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:15 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3295115296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:15.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:15 smithi102 ceph-mon[41371]: from='client.46802 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:15 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:15 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:05:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:15 smithi146 ceph-mon[33158]: pgmap v248: 268 pgs: 40 unknown, 32 creating+peering, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:05:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:15 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:05:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:15 smithi146 ceph-mon[33158]: osdmap e187: 8 total, 8 up, 8 in 2024-09-11T08:05:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:15 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1488050306' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi102-80356-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:15 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-21"}]: dispatch 2024-09-11T08:05:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:15 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-21"}]: dispatch 2024-09-11T08:05:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:15 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3295115296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:15 smithi146 ceph-mon[33158]: from='client.46802 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:16.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:16 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:05:16.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1488050306' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi102-80356-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:16.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:16 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-21"}]': finished 2024-09-11T08:05:16.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:16 smithi102 ceph-mon[41371]: from='client.46802 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi102-80338-32"}]': finished 2024-09-11T08:05:16.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:16 smithi102 ceph-mon[41371]: osdmap e188: 8 total, 8 up, 8 in 2024-09-11T08:05:16.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3295115296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:16.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:16 smithi102 ceph-mon[41371]: from='client.46802 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:16 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:05:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1488050306' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi102-80356-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:16 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-21"}]': finished 2024-09-11T08:05:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:16 smithi102 ceph-mon[29849]: from='client.46802 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi102-80338-32"}]': finished 2024-09-11T08:05:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:16 smithi102 ceph-mon[29849]: osdmap e188: 8 total, 8 up, 8 in 2024-09-11T08:05:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3295115296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:16.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:16 smithi102 ceph-mon[29849]: from='client.46802 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:16.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:16 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:05:16.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1488050306' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi102-80356-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:16.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:16 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-21"}]': finished 2024-09-11T08:05:16.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:16 smithi146 ceph-mon[33158]: from='client.46802 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi102-80338-32"}]': finished 2024-09-11T08:05:16.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:16 smithi146 ceph-mon[33158]: osdmap e188: 8 total, 8 up, 8 in 2024-09-11T08:05:16.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3295115296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:16.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:16 smithi146 ceph-mon[33158]: from='client.46802 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi102-80338-32"}]: dispatch 2024-09-11T08:05:17.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[41371]: pgmap v251: 292 pgs: 32 creating+peering, 260 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-11T08:05:17.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:17.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[41371]: from='client.46802 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi102-80338-32"}]': finished 2024-09-11T08:05:17.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[41371]: osdmap e189: 8 total, 8 up, 8 in 2024-09-11T08:05:17.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2676608989' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi102-80356-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:17.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[41371]: from='client.46820 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi102-80356-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:17.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1615011055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:17.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[41371]: from='client.46826 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:17.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1615011055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:17.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[41371]: from='client.46826 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:17.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1615011055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi102-80338-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:17.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[41371]: from='client.46826 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi102-80338-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:17.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[29849]: pgmap v251: 292 pgs: 32 creating+peering, 260 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-11T08:05:17.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:17.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[29849]: from='client.46802 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi102-80338-32"}]': finished 2024-09-11T08:05:17.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[29849]: osdmap e189: 8 total, 8 up, 8 in 2024-09-11T08:05:17.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2676608989' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi102-80356-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:17.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[29849]: from='client.46820 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi102-80356-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:17.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1615011055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:17.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[29849]: from='client.46826 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:17.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1615011055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:17.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[29849]: from='client.46826 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:17.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1615011055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi102-80338-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:17.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:17 smithi102 ceph-mon[29849]: from='client.46826 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi102-80338-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:17.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:17 smithi146 ceph-mon[33158]: pgmap v251: 292 pgs: 32 creating+peering, 260 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-11T08:05:17.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:17 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:17.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:17 smithi146 ceph-mon[33158]: from='client.46802 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi102-80338-32"}]': finished 2024-09-11T08:05:17.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:17 smithi146 ceph-mon[33158]: osdmap e189: 8 total, 8 up, 8 in 2024-09-11T08:05:17.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2676608989' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi102-80356-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:17.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:17 smithi146 ceph-mon[33158]: from='client.46820 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi102-80356-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:17.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1615011055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:17.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:17 smithi146 ceph-mon[33158]: from='client.46826 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:17.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1615011055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:17.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:17 smithi146 ceph-mon[33158]: from='client.46826 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:17.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1615011055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi102-80338-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:17.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:17 smithi146 ceph-mon[33158]: from='client.46826 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi102-80338-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:19.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:19 smithi102 ceph-mon[41371]: from='client.46820 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi102-80356-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:19.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:19 smithi102 ceph-mon[41371]: from='client.46826 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi102-80338-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:19.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:19 smithi102 ceph-mon[41371]: osdmap e190: 8 total, 8 up, 8 in 2024-09-11T08:05:19.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1615011055' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi102-80338-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:19.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:19.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:19 smithi102 ceph-mon[41371]: from='client.46826 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi102-80338-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:19.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:19 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:19.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:19 smithi102 ceph-mon[41371]: pgmap v254: 324 pgs: 64 unknown, 32 creating+peering, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:05:19.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:19 smithi102 ceph-mon[29849]: from='client.46820 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi102-80356-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:19.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:19 smithi102 ceph-mon[29849]: from='client.46826 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi102-80338-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:19.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:19 smithi102 ceph-mon[29849]: osdmap e190: 8 total, 8 up, 8 in 2024-09-11T08:05:19.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1615011055' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi102-80338-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:19.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:19.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:19 smithi102 ceph-mon[29849]: from='client.46826 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi102-80338-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:19.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:19 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:19.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:19 smithi102 ceph-mon[29849]: pgmap v254: 324 pgs: 64 unknown, 32 creating+peering, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:05:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:19 smithi146 ceph-mon[33158]: from='client.46820 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi102-80356-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:19 smithi146 ceph-mon[33158]: from='client.46826 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi102-80338-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:19 smithi146 ceph-mon[33158]: osdmap e190: 8 total, 8 up, 8 in 2024-09-11T08:05:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1615011055' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi102-80338-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:19 smithi146 ceph-mon[33158]: from='client.46826 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi102-80338-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:19 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:19 smithi146 ceph-mon[33158]: pgmap v254: 324 pgs: 64 unknown, 32 creating+peering, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:05:19.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:05:19 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:05:20.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:05:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:05:19] "GET /metrics HTTP/1.1" 200 233322 "" "Prometheus/2.43.0" 2024-09-11T08:05:20.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:20 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:20.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:20 smithi102 ceph-mon[41371]: osdmap e191: 8 total, 8 up, 8 in 2024-09-11T08:05:20.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:20 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:05:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:20 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:20 smithi102 ceph-mon[29849]: osdmap e191: 8 total, 8 up, 8 in 2024-09-11T08:05:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:20 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:05:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:20 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:20 smithi146 ceph-mon[33158]: osdmap e191: 8 total, 8 up, 8 in 2024-09-11T08:05:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:20 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:05:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:21 smithi102 ceph-mon[29849]: from='client.46826 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi102-80338-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi102-80338-33"}]': finished 2024-09-11T08:05:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:21 smithi102 ceph-mon[29849]: osdmap e192: 8 total, 8 up, 8 in 2024-09-11T08:05:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:21 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:21 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:21 smithi102 ceph-mon[29849]: pgmap v257: 268 pgs: 8 creating+peering, 260 active+clean; 584 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-09-11T08:05:21.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:21 smithi102 ceph-mon[41371]: from='client.46826 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi102-80338-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi102-80338-33"}]': finished 2024-09-11T08:05:21.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:21 smithi102 ceph-mon[41371]: osdmap e192: 8 total, 8 up, 8 in 2024-09-11T08:05:21.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:21 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:21.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:21 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:21.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:21 smithi102 ceph-mon[41371]: pgmap v257: 268 pgs: 8 creating+peering, 260 active+clean; 584 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-09-11T08:05:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:21 smithi146 ceph-mon[33158]: from='client.46826 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi102-80338-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi102-80338-33"}]': finished 2024-09-11T08:05:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:21 smithi146 ceph-mon[33158]: osdmap e192: 8 total, 8 up, 8 in 2024-09-11T08:05:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:21 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:21 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:21 smithi146 ceph-mon[33158]: pgmap v257: 268 pgs: 8 creating+peering, 260 active+clean; 584 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-09-11T08:05:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:22 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:05:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:22 smithi102 ceph-mon[41371]: osdmap e193: 8 total, 8 up, 8 in 2024-09-11T08:05:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-23"}]: dispatch 2024-09-11T08:05:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:22 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-23"}]: dispatch 2024-09-11T08:05:22.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:22 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:05:22.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:22 smithi102 ceph-mon[29849]: osdmap e193: 8 total, 8 up, 8 in 2024-09-11T08:05:22.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-23"}]: dispatch 2024-09-11T08:05:22.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:22 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-23"}]: dispatch 2024-09-11T08:05:22.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:22 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:05:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:22 smithi146 ceph-mon[33158]: osdmap e193: 8 total, 8 up, 8 in 2024-09-11T08:05:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-23"}]: dispatch 2024-09-11T08:05:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:22 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-23"}]: dispatch 2024-09-11T08:05:23.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:23 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-23"}]': finished 2024-09-11T08:05:23.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:23 smithi102 ceph-mon[41371]: osdmap e194: 8 total, 8 up, 8 in 2024-09-11T08:05:23.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3698787814' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:23.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1615011055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:23.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-23", "mode": "writeback"}]: dispatch 2024-09-11T08:05:23.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:23 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-23", "mode": "writeback"}]: dispatch 2024-09-11T08:05:23.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:23 smithi102 ceph-mon[41371]: from='client.46826 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:23.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:23 smithi102 ceph-mon[41371]: pgmap v260: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:05:23.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:23 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:05:23.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3698787814' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:23.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:23 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-23", "mode": "writeback"}]': finished 2024-09-11T08:05:23.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:23 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-23"}]': finished 2024-09-11T08:05:23.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:23 smithi102 ceph-mon[29849]: osdmap e194: 8 total, 8 up, 8 in 2024-09-11T08:05:23.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3698787814' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:23.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1615011055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:23.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-23", "mode": "writeback"}]: dispatch 2024-09-11T08:05:23.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:23 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-23", "mode": "writeback"}]: dispatch 2024-09-11T08:05:23.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:23 smithi102 ceph-mon[29849]: from='client.46826 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:23.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:23 smithi102 ceph-mon[29849]: pgmap v260: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:05:23.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:23 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:05:23.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3698787814' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:23.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:23 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-23", "mode": "writeback"}]': finished 2024-09-11T08:05:23.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:23 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-23"}]': finished 2024-09-11T08:05:23.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:23 smithi146 ceph-mon[33158]: osdmap e194: 8 total, 8 up, 8 in 2024-09-11T08:05:23.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3698787814' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:23.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1615011055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:23.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-23", "mode": "writeback"}]: dispatch 2024-09-11T08:05:23.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:23 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-23", "mode": "writeback"}]: dispatch 2024-09-11T08:05:23.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:23 smithi146 ceph-mon[33158]: from='client.46826 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:23.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:23 smithi146 ceph-mon[33158]: pgmap v260: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-11T08:05:23.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:23 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:05:23.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3698787814' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:23.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:23 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-23", "mode": "writeback"}]': finished 2024-09-11T08:05:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:24 smithi102 ceph-mon[41371]: from='client.46826 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi102-80338-33"}]': finished 2024-09-11T08:05:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:24 smithi102 ceph-mon[41371]: osdmap e195: 8 total, 8 up, 8 in 2024-09-11T08:05:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1615011055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:24 smithi102 ceph-mon[41371]: from='client.46826 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:24 smithi102 ceph-mon[41371]: from='client.46826 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi102-80338-33"}]': finished 2024-09-11T08:05:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:24 smithi102 ceph-mon[41371]: osdmap e196: 8 total, 8 up, 8 in 2024-09-11T08:05:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2744181827' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:24.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:24 smithi102 ceph-mon[29849]: from='client.46826 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi102-80338-33"}]': finished 2024-09-11T08:05:24.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:24 smithi102 ceph-mon[29849]: osdmap e195: 8 total, 8 up, 8 in 2024-09-11T08:05:24.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1615011055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:24.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:24 smithi102 ceph-mon[29849]: from='client.46826 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:24.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:24 smithi102 ceph-mon[29849]: from='client.46826 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi102-80338-33"}]': finished 2024-09-11T08:05:24.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:24 smithi102 ceph-mon[29849]: osdmap e196: 8 total, 8 up, 8 in 2024-09-11T08:05:24.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2744181827' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:24 smithi146 ceph-mon[33158]: from='client.46826 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi102-80338-33"}]': finished 2024-09-11T08:05:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:24 smithi146 ceph-mon[33158]: osdmap e195: 8 total, 8 up, 8 in 2024-09-11T08:05:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1615011055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:24 smithi146 ceph-mon[33158]: from='client.46826 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi102-80338-33"}]: dispatch 2024-09-11T08:05:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:24 smithi146 ceph-mon[33158]: from='client.46826 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi102-80338-33"}]': finished 2024-09-11T08:05:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:24 smithi146 ceph-mon[33158]: osdmap e196: 8 total, 8 up, 8 in 2024-09-11T08:05:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2744181827' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:24.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:05:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:05:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:05:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2145136644' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[29849]: from='client.47944 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2145136644' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[29849]: from='client.47944 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2145136644' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi102-80338-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[29849]: from='client.47944 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi102-80338-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[29849]: pgmap v263: 292 pgs: 64 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:05:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2744181827' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:05:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[29849]: from='client.47944 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi102-80338-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[29849]: osdmap e197: 8 total, 8 up, 8 in 2024-09-11T08:05:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-23"}]: dispatch 2024-09-11T08:05:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2145136644' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi102-80338-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-23"}]: dispatch 2024-09-11T08:05:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[29849]: from='client.47944 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi102-80338-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:25.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:25.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:25.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2145136644' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:25.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[41371]: from='client.47944 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:25.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2145136644' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:25.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[41371]: from='client.47944 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:25.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2145136644' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi102-80338-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:25.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[41371]: from='client.47944 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi102-80338-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:25.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[41371]: pgmap v263: 292 pgs: 64 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:05:25.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2744181827' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:25.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:05:25.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[41371]: from='client.47944 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi102-80338-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:25.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[41371]: osdmap e197: 8 total, 8 up, 8 in 2024-09-11T08:05:25.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-23"}]: dispatch 2024-09-11T08:05:25.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2145136644' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi102-80338-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:25.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-23"}]: dispatch 2024-09-11T08:05:25.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:25 smithi102 ceph-mon[41371]: from='client.47944 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi102-80338-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:25 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2145136644' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:25 smithi146 ceph-mon[33158]: from='client.47944 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2145136644' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:25 smithi146 ceph-mon[33158]: from='client.47944 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2145136644' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi102-80338-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:25 smithi146 ceph-mon[33158]: from='client.47944 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi102-80338-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:25 smithi146 ceph-mon[33158]: pgmap v263: 292 pgs: 64 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:05:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2744181827' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:25 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:05:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:25 smithi146 ceph-mon[33158]: from='client.47944 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi102-80338-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:25 smithi146 ceph-mon[33158]: osdmap e197: 8 total, 8 up, 8 in 2024-09-11T08:05:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-23"}]: dispatch 2024-09-11T08:05:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2145136644' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi102-80338-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:25 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-23"}]: dispatch 2024-09-11T08:05:25.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:25 smithi146 ceph-mon[33158]: from='client.47944 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi102-80338-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:26.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:26 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:05:26.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:26 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:05:26.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:26 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:05:27.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:27 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-23"}]': finished 2024-09-11T08:05:27.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:27 smithi102 ceph-mon[29849]: osdmap e198: 8 total, 8 up, 8 in 2024-09-11T08:05:27.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2833335892' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:27.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:27 smithi102 ceph-mon[29849]: pgmap v266: 324 pgs: 32 unknown, 292 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-11T08:05:27.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:27 smithi102 ceph-mon[29849]: from='client.47944 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi102-80338-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi102-80338-34"}]': finished 2024-09-11T08:05:27.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2833335892' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:27.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:27 smithi102 ceph-mon[29849]: osdmap e199: 8 total, 8 up, 8 in 2024-09-11T08:05:27.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:27 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-23"}]': finished 2024-09-11T08:05:27.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:27 smithi102 ceph-mon[41371]: osdmap e198: 8 total, 8 up, 8 in 2024-09-11T08:05:27.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2833335892' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:27.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:27 smithi102 ceph-mon[41371]: pgmap v266: 324 pgs: 32 unknown, 292 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-11T08:05:27.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:27 smithi102 ceph-mon[41371]: from='client.47944 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi102-80338-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi102-80338-34"}]': finished 2024-09-11T08:05:27.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2833335892' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:27.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:27 smithi102 ceph-mon[41371]: osdmap e199: 8 total, 8 up, 8 in 2024-09-11T08:05:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:27 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-23"}]': finished 2024-09-11T08:05:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:27 smithi146 ceph-mon[33158]: osdmap e198: 8 total, 8 up, 8 in 2024-09-11T08:05:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2833335892' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:27 smithi146 ceph-mon[33158]: pgmap v266: 324 pgs: 32 unknown, 292 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-11T08:05:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:27 smithi146 ceph-mon[33158]: from='client.47944 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi102-80338-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi102-80338-34"}]': finished 2024-09-11T08:05:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2833335892' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:27 smithi146 ceph-mon[33158]: osdmap e199: 8 total, 8 up, 8 in 2024-09-11T08:05:29.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:29 smithi102 ceph-mon[41371]: osdmap e200: 8 total, 8 up, 8 in 2024-09-11T08:05:29.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/701752039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:29.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:29 smithi102 ceph-mon[41371]: from='client.46856 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:29.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:29.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:29 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:29.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:29 smithi102 ceph-mon[41371]: pgmap v269: 364 pgs: 104 unknown, 260 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:05:29.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:29 smithi102 ceph-mon[29849]: osdmap e200: 8 total, 8 up, 8 in 2024-09-11T08:05:29.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/701752039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:29.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:29 smithi102 ceph-mon[29849]: from='client.46856 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:29.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:29.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:29 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:29.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:29 smithi102 ceph-mon[29849]: pgmap v269: 364 pgs: 104 unknown, 260 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:05:29.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:29 smithi146 ceph-mon[33158]: osdmap e200: 8 total, 8 up, 8 in 2024-09-11T08:05:29.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/701752039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:29.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:29 smithi146 ceph-mon[33158]: from='client.46856 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:29.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:29.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:29 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:29.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:29 smithi146 ceph-mon[33158]: pgmap v269: 364 pgs: 104 unknown, 260 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:05:29.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:05:29 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:05:30.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:05:29 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:05:29] "GET /metrics HTTP/1.1" 200 238175 "" "Prometheus/2.43.0" 2024-09-11T08:05:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:30 smithi102 ceph-mon[41371]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:30 smithi102 ceph-mon[41371]: from='client.46856 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:30 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:30 smithi102 ceph-mon[41371]: osdmap e201: 8 total, 8 up, 8 in 2024-09-11T08:05:30.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:30 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2145136644' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:30.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:30 smithi102 ceph-mon[41371]: from='client.47944 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:30.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:30 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:05:30.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:30 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:05:30.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:30 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:05:30.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:30 smithi102 ceph-mon[29849]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:30.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:30 smithi102 ceph-mon[29849]: from='client.46856 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:30.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:30 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:30.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:30 smithi102 ceph-mon[29849]: osdmap e201: 8 total, 8 up, 8 in 2024-09-11T08:05:30.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:30 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2145136644' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:30.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:30 smithi102 ceph-mon[29849]: from='client.47944 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:30.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:30 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:05:30.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:30 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:05:30.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:30 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:05:30.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:30 smithi146 ceph-mon[33158]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:30.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:30 smithi146 ceph-mon[33158]: from='client.46856 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:30.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:30 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:30.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:30 smithi146 ceph-mon[33158]: osdmap e201: 8 total, 8 up, 8 in 2024-09-11T08:05:30.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:30 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2145136644' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:30.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:30 smithi146 ceph-mon[33158]: from='client.47944 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:30.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:30 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:05:30.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:30 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:05:30.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:30 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:05:31.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:31 smithi102 ceph-mon[29849]: from='client.47944 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi102-80338-34"}]': finished 2024-09-11T08:05:31.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:31 smithi102 ceph-mon[29849]: osdmap e202: 8 total, 8 up, 8 in 2024-09-11T08:05:31.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:31 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/588252610' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:31.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:31 smithi102 ceph-mon[29849]: from='client.46862 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:31.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:31 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2145136644' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:31.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:31 smithi102 ceph-mon[29849]: from='client.47944 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:31.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:31 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:31.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:31 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:31.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:31 smithi102 ceph-mon[29849]: pgmap v272: 388 pgs: 32 creating+peering, 356 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:05:31.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:31 smithi102 ceph-mon[41371]: from='client.47944 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi102-80338-34"}]': finished 2024-09-11T08:05:31.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:31 smithi102 ceph-mon[41371]: osdmap e202: 8 total, 8 up, 8 in 2024-09-11T08:05:31.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:31 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/588252610' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:31.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:31 smithi102 ceph-mon[41371]: from='client.46862 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:31.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:31 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2145136644' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:31.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:31 smithi102 ceph-mon[41371]: from='client.47944 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:31.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:31 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:31.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:31 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:31.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:31 smithi102 ceph-mon[41371]: pgmap v272: 388 pgs: 32 creating+peering, 356 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:05:31.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:31 smithi146 ceph-mon[33158]: from='client.47944 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi102-80338-34"}]': finished 2024-09-11T08:05:31.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:31 smithi146 ceph-mon[33158]: osdmap e202: 8 total, 8 up, 8 in 2024-09-11T08:05:31.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:31 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/588252610' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:31.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:31 smithi146 ceph-mon[33158]: from='client.46862 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:31.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:31 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2145136644' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:31.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:31 smithi146 ceph-mon[33158]: from='client.47944 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi102-80338-34"}]: dispatch 2024-09-11T08:05:31.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:31 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:31.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:31 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:31.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:31 smithi146 ceph-mon[33158]: pgmap v272: 388 pgs: 32 creating+peering, 356 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:05:32.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[29849]: from='client.46862 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:32.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[29849]: from='client.47944 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi102-80338-34"}]': finished 2024-09-11T08:05:32.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:05:32.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[29849]: osdmap e203: 8 total, 8 up, 8 in 2024-09-11T08:05:32.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-25"}]: dispatch 2024-09-11T08:05:32.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-25"}]: dispatch 2024-09-11T08:05:32.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi102-80338-35"}]: dispatch 2024-09-11T08:05:32.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi102-80338-35"}]: dispatch 2024-09-11T08:05:32.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi102-80338-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:32.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-25"}]': finished 2024-09-11T08:05:32.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi102-80338-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:32.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[29849]: osdmap e204: 8 total, 8 up, 8 in 2024-09-11T08:05:32.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-25", "mode": "writeback"}]: dispatch 2024-09-11T08:05:32.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi102-80338-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi102-80338-35"}]: dispatch 2024-09-11T08:05:32.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-25", "mode": "writeback"}]: dispatch 2024-09-11T08:05:32.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[41371]: from='client.46862 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:32.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[41371]: from='client.47944 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi102-80338-34"}]': finished 2024-09-11T08:05:32.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:05:32.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[41371]: osdmap e203: 8 total, 8 up, 8 in 2024-09-11T08:05:32.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-25"}]: dispatch 2024-09-11T08:05:32.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-25"}]: dispatch 2024-09-11T08:05:32.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi102-80338-35"}]: dispatch 2024-09-11T08:05:32.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi102-80338-35"}]: dispatch 2024-09-11T08:05:32.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi102-80338-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:32.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-25"}]': finished 2024-09-11T08:05:32.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi102-80338-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:32.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[41371]: osdmap e204: 8 total, 8 up, 8 in 2024-09-11T08:05:32.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-25", "mode": "writeback"}]: dispatch 2024-09-11T08:05:32.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi102-80338-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi102-80338-35"}]: dispatch 2024-09-11T08:05:32.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:32 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-25", "mode": "writeback"}]: dispatch 2024-09-11T08:05:32.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:32 smithi146 ceph-mon[33158]: from='client.46862 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi102-80356-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:32.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:32 smithi146 ceph-mon[33158]: from='client.47944 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi102-80338-34"}]': finished 2024-09-11T08:05:32.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:32 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:05:32.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:32 smithi146 ceph-mon[33158]: osdmap e203: 8 total, 8 up, 8 in 2024-09-11T08:05:32.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-25"}]: dispatch 2024-09-11T08:05:32.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:32 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-25"}]: dispatch 2024-09-11T08:05:32.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi102-80338-35"}]: dispatch 2024-09-11T08:05:32.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi102-80338-35"}]: dispatch 2024-09-11T08:05:32.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi102-80338-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:32.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:32 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-25"}]': finished 2024-09-11T08:05:32.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi102-80338-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:32.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:32 smithi146 ceph-mon[33158]: osdmap e204: 8 total, 8 up, 8 in 2024-09-11T08:05:32.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-25", "mode": "writeback"}]: dispatch 2024-09-11T08:05:32.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi102-80338-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi102-80338-35"}]: dispatch 2024-09-11T08:05:32.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:32 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-25", "mode": "writeback"}]: dispatch 2024-09-11T08:05:33.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:33 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:05:33.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:33 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:05:33.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:33 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:05:34.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:34 smithi102 ceph-mon[41371]: pgmap v275: 356 pgs: 356 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:05:34.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:34 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-25", "mode": "writeback"}]': finished 2024-09-11T08:05:34.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:34 smithi102 ceph-mon[41371]: osdmap e205: 8 total, 8 up, 8 in 2024-09-11T08:05:34.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:34 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:34.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:34 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:34.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:34 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi102-80338-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi102-80338-35"}]': finished 2024-09-11T08:05:34.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:34 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:05:34.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:34 smithi102 ceph-mon[41371]: osdmap e206: 8 total, 8 up, 8 in 2024-09-11T08:05:34.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:34 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-25"}]: dispatch 2024-09-11T08:05:34.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:34 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-25"}]: dispatch 2024-09-11T08:05:34.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:34 smithi102 ceph-mon[29849]: pgmap v275: 356 pgs: 356 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:05:34.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:34 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-25", "mode": "writeback"}]': finished 2024-09-11T08:05:34.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:34 smithi102 ceph-mon[29849]: osdmap e205: 8 total, 8 up, 8 in 2024-09-11T08:05:34.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:34 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:34.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:34 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:34.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:34 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi102-80338-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi102-80338-35"}]': finished 2024-09-11T08:05:34.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:34 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:05:34.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:34 smithi102 ceph-mon[29849]: osdmap e206: 8 total, 8 up, 8 in 2024-09-11T08:05:34.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:34 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-25"}]: dispatch 2024-09-11T08:05:34.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:34 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-25"}]: dispatch 2024-09-11T08:05:34.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:34 smithi146 ceph-mon[33158]: pgmap v275: 356 pgs: 356 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:05:34.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:34 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-25", "mode": "writeback"}]': finished 2024-09-11T08:05:34.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:34 smithi146 ceph-mon[33158]: osdmap e205: 8 total, 8 up, 8 in 2024-09-11T08:05:34.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:34 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:34.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:34 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:34.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:34 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi102-80338-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi102-80338-35"}]': finished 2024-09-11T08:05:34.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:34 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:05:34.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:34 smithi146 ceph-mon[33158]: osdmap e206: 8 total, 8 up, 8 in 2024-09-11T08:05:34.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:34 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-25"}]: dispatch 2024-09-11T08:05:34.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:34 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-25"}]: dispatch 2024-09-11T08:05:34.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:05:34 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:05:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:05:35.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:35 smithi102 ceph-mon[41371]: pgmap v278: 300 pgs: 8 unknown, 292 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:05:35.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:35 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:05:35.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:35 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-25"}]': finished 2024-09-11T08:05:35.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:35 smithi102 ceph-mon[41371]: osdmap e207: 8 total, 8 up, 8 in 2024-09-11T08:05:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:35 smithi102 ceph-mon[29849]: pgmap v278: 300 pgs: 8 unknown, 292 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:05:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:35 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:05:35.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:35 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-25"}]': finished 2024-09-11T08:05:35.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:35 smithi102 ceph-mon[29849]: osdmap e207: 8 total, 8 up, 8 in 2024-09-11T08:05:35.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:35 smithi146 ceph-mon[33158]: pgmap v278: 300 pgs: 8 unknown, 292 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:05:35.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:35 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:05:35.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:35 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-25"}]': finished 2024-09-11T08:05:35.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:35 smithi146 ceph-mon[33158]: osdmap e207: 8 total, 8 up, 8 in 2024-09-11T08:05:36.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:36 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:36.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:36 smithi102 ceph-mon[29849]: osdmap e208: 8 total, 8 up, 8 in 2024-09-11T08:05:36.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:36 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi102-80338-35"}]: dispatch 2024-09-11T08:05:36.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:36 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:36.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:36 smithi102 ceph-mon[41371]: osdmap e208: 8 total, 8 up, 8 in 2024-09-11T08:05:36.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:36 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi102-80338-35"}]: dispatch 2024-09-11T08:05:36.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:36 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:36.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:36 smithi146 ceph-mon[33158]: osdmap e208: 8 total, 8 up, 8 in 2024-09-11T08:05:36.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:36 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi102-80338-35"}]: dispatch 2024-09-11T08:05:38.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:37 smithi146 ceph-mon[33158]: pgmap v281: 196 pgs: 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:05:38.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi102-80338-35"}]': finished 2024-09-11T08:05:38.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:37 smithi146 ceph-mon[33158]: osdmap e209: 8 total, 8 up, 8 in 2024-09-11T08:05:38.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3671033824' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi102-80356-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:38.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:38.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi102-80338-35"}]: dispatch 2024-09-11T08:05:38.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:37 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:38.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:37 smithi146 ceph-mon[33158]: from='client.46874 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi102-80356-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:38.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:37 smithi102 ceph-mon[29849]: pgmap v281: 196 pgs: 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:05:38.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi102-80338-35"}]': finished 2024-09-11T08:05:38.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:37 smithi102 ceph-mon[29849]: osdmap e209: 8 total, 8 up, 8 in 2024-09-11T08:05:38.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3671033824' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi102-80356-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:38.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:38.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi102-80338-35"}]: dispatch 2024-09-11T08:05:38.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:37 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:38.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:37 smithi102 ceph-mon[29849]: from='client.46874 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi102-80356-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:38.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:37 smithi102 ceph-mon[41371]: pgmap v281: 196 pgs: 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:05:38.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi102-80338-35"}]': finished 2024-09-11T08:05:38.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:37 smithi102 ceph-mon[41371]: osdmap e209: 8 total, 8 up, 8 in 2024-09-11T08:05:38.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3671033824' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi102-80356-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:38.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:38.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi102-80338-35"}]: dispatch 2024-09-11T08:05:38.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:37 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:38.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:37 smithi102 ceph-mon[41371]: from='client.46874 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi102-80356-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi102-80338-35"}]': finished 2024-09-11T08:05:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:38 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:38 smithi146 ceph-mon[33158]: from='client.46874 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi102-80356-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:38 smithi146 ceph-mon[33158]: osdmap e210: 8 total, 8 up, 8 in 2024-09-11T08:05:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4177807146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:38 smithi146 ceph-mon[33158]: from='client.47980 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4177807146' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:38 smithi146 ceph-mon[33158]: from='client.47980 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:39.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4177807146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:39.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:38 smithi146 ceph-mon[33158]: from='client.47980 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:39.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:38 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:05:39.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi102-80338-35"}]': finished 2024-09-11T08:05:39.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:38 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:39.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:38 smithi102 ceph-mon[41371]: from='client.46874 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi102-80356-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:39.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:38 smithi102 ceph-mon[41371]: osdmap e210: 8 total, 8 up, 8 in 2024-09-11T08:05:39.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4177807146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:39.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:38 smithi102 ceph-mon[41371]: from='client.47980 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:39.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4177807146' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:39.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:38 smithi102 ceph-mon[41371]: from='client.47980 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:39.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4177807146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:39.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:38 smithi102 ceph-mon[41371]: from='client.47980 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:39.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:38 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:05:39.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3005892145' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi102-80338-35"}]': finished 2024-09-11T08:05:39.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:38 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:39.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:38 smithi102 ceph-mon[29849]: from='client.46874 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi102-80356-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:39.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:38 smithi102 ceph-mon[29849]: osdmap e210: 8 total, 8 up, 8 in 2024-09-11T08:05:39.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4177807146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:39.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:38 smithi102 ceph-mon[29849]: from='client.47980 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:39.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4177807146' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:39.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:38 smithi102 ceph-mon[29849]: from='client.47980 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:39.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4177807146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:39.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:38 smithi102 ceph-mon[29849]: from='client.47980 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:39.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:38 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:05:39.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:05:39 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:05:40.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:05:39 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:05:39] "GET /metrics HTTP/1.1" 200 238175 "" "Prometheus/2.43.0" 2024-09-11T08:05:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:40 smithi146 ceph-mon[33158]: pgmap v284: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:05:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:40 smithi146 ceph-mon[33158]: from='client.47980 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:40 smithi146 ceph-mon[33158]: osdmap e211: 8 total, 8 up, 8 in 2024-09-11T08:05:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4177807146' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi102-80338-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:40 smithi146 ceph-mon[33158]: from='client.47980 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi102-80338-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:40 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:40 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:40.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:40 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:05:40.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:40 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:05:40.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:40 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:05:40.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:40 smithi102 ceph-mon[29849]: pgmap v284: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:05:40.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:40 smithi102 ceph-mon[29849]: from='client.47980 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:40.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:40 smithi102 ceph-mon[29849]: osdmap e211: 8 total, 8 up, 8 in 2024-09-11T08:05:40.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4177807146' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi102-80338-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:40.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:40 smithi102 ceph-mon[29849]: from='client.47980 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi102-80338-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:40.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:40.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:40 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:40.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:40 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:40.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:40 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:05:40.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:40 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:05:40.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:40 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:05:40.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:40 smithi102 ceph-mon[41371]: pgmap v284: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:05:40.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:40 smithi102 ceph-mon[41371]: from='client.47980 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:40.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:40 smithi102 ceph-mon[41371]: osdmap e211: 8 total, 8 up, 8 in 2024-09-11T08:05:40.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4177807146' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi102-80338-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:40.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:40 smithi102 ceph-mon[41371]: from='client.47980 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi102-80338-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:40.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:40.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:40 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:40.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:40 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:05:40.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:40 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:05:40.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:40 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:05:40.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:40 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:05:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:41 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:05:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:41 smithi146 ceph-mon[33158]: osdmap e212: 8 total, 8 up, 8 in 2024-09-11T08:05:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-27"}]: dispatch 2024-09-11T08:05:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3868799677' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi102-80356-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:41 smithi146 ceph-mon[33158]: from='client.46883 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi102-80356-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:41 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-27"}]: dispatch 2024-09-11T08:05:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:41 smithi146 ceph-mon[33158]: pgmap v287: 260 pgs: 32 creating+peering, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:05:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:41 smithi146 ceph-mon[33158]: from='client.47980 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi102-80338-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi102-80338-36"}]': finished 2024-09-11T08:05:41.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:41 smithi146 ceph-mon[33158]: from='client.46883 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi102-80356-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:41.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:41 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-27"}]': finished 2024-09-11T08:05:41.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:41 smithi146 ceph-mon[33158]: osdmap e213: 8 total, 8 up, 8 in 2024-09-11T08:05:41.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-27", "mode": "writeback"}]: dispatch 2024-09-11T08:05:41.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:41 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-27", "mode": "writeback"}]: dispatch 2024-09-11T08:05:41.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:05:41.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[29849]: osdmap e212: 8 total, 8 up, 8 in 2024-09-11T08:05:41.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-27"}]: dispatch 2024-09-11T08:05:41.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3868799677' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi102-80356-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:41.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[29849]: from='client.46883 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi102-80356-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:41.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-27"}]: dispatch 2024-09-11T08:05:41.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[29849]: pgmap v287: 260 pgs: 32 creating+peering, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:05:41.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[29849]: from='client.47980 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi102-80338-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi102-80338-36"}]': finished 2024-09-11T08:05:41.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[29849]: from='client.46883 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi102-80356-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:41.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-27"}]': finished 2024-09-11T08:05:41.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[29849]: osdmap e213: 8 total, 8 up, 8 in 2024-09-11T08:05:41.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-27", "mode": "writeback"}]: dispatch 2024-09-11T08:05:41.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-27", "mode": "writeback"}]: dispatch 2024-09-11T08:05:41.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:05:41.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[41371]: osdmap e212: 8 total, 8 up, 8 in 2024-09-11T08:05:41.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-27"}]: dispatch 2024-09-11T08:05:41.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3868799677' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi102-80356-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:41.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[41371]: from='client.46883 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi102-80356-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:41.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-27"}]: dispatch 2024-09-11T08:05:41.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[41371]: pgmap v287: 260 pgs: 32 creating+peering, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:05:41.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[41371]: from='client.47980 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi102-80338-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi102-80338-36"}]': finished 2024-09-11T08:05:41.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[41371]: from='client.46883 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi102-80356-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:41.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-27"}]': finished 2024-09-11T08:05:41.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[41371]: osdmap e213: 8 total, 8 up, 8 in 2024-09-11T08:05:41.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-27", "mode": "writeback"}]: dispatch 2024-09-11T08:05:41.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:41 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-27", "mode": "writeback"}]: dispatch 2024-09-11T08:05:42.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:42 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:42.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:42 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:05:42.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:42 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-27", "mode": "writeback"}]': finished 2024-09-11T08:05:42.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:42 smithi146 ceph-mon[33158]: osdmap e214: 8 total, 8 up, 8 in 2024-09-11T08:05:42.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1738569373' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:42.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:42 smithi146 ceph-mon[33158]: from='client.46889 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:42.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1738569373' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:42.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:42 smithi146 ceph-mon[33158]: from='client.46889 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:42.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1738569373' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi102-80356-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:42.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:42 smithi146 ceph-mon[33158]: from='client.46889 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi102-80356-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:42.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:42 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:42.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:42 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:05:42.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:42 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-27", "mode": "writeback"}]': finished 2024-09-11T08:05:42.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:42 smithi102 ceph-mon[29849]: osdmap e214: 8 total, 8 up, 8 in 2024-09-11T08:05:42.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1738569373' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:42.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:42 smithi102 ceph-mon[29849]: from='client.46889 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:42.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1738569373' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:42.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:42 smithi102 ceph-mon[29849]: from='client.46889 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:42.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1738569373' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi102-80356-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:42.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:42 smithi102 ceph-mon[29849]: from='client.46889 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi102-80356-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:42.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:42 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:42.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:42 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:05:42.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:42 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-27", "mode": "writeback"}]': finished 2024-09-11T08:05:42.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:42 smithi102 ceph-mon[41371]: osdmap e214: 8 total, 8 up, 8 in 2024-09-11T08:05:42.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1738569373' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:42.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:42 smithi102 ceph-mon[41371]: from='client.46889 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:42.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1738569373' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:42.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:42 smithi102 ceph-mon[41371]: from='client.46889 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:42.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1738569373' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi102-80356-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:42.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:42 smithi102 ceph-mon[41371]: from='client.46889 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi102-80356-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:43 smithi146 ceph-mon[33158]: pgmap v290: 236 pgs: 8 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:05:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:43 smithi146 ceph-mon[33158]: from='client.46889 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi102-80356-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:43 smithi146 ceph-mon[33158]: osdmap e215: 8 total, 8 up, 8 in 2024-09-11T08:05:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:43 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1738569373' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi102-80356-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:43 smithi146 ceph-mon[33158]: from='client.46889 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi102-80356-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:43 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4177807146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:43 smithi146 ceph-mon[33158]: from='client.47980 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:43 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:43.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:43 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:43.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:43 smithi102 ceph-mon[41371]: pgmap v290: 236 pgs: 8 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:05:43.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:43 smithi102 ceph-mon[41371]: from='client.46889 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi102-80356-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:43.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:43 smithi102 ceph-mon[41371]: osdmap e215: 8 total, 8 up, 8 in 2024-09-11T08:05:43.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:43 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1738569373' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi102-80356-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:43.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:43 smithi102 ceph-mon[41371]: from='client.46889 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi102-80356-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:43.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:43 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4177807146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:43.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:43 smithi102 ceph-mon[41371]: from='client.47980 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:43.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:43 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:43.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:43 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:43.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:43 smithi102 ceph-mon[29849]: pgmap v290: 236 pgs: 8 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:05:43.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:43 smithi102 ceph-mon[29849]: from='client.46889 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi102-80356-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:43.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:43 smithi102 ceph-mon[29849]: osdmap e215: 8 total, 8 up, 8 in 2024-09-11T08:05:43.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:43 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1738569373' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi102-80356-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:43.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:43 smithi102 ceph-mon[29849]: from='client.46889 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi102-80356-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:43.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:43 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4177807146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:43.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:43 smithi102 ceph-mon[29849]: from='client.47980 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:43.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:43 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:43.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:43 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:44.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:05:44 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:05:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:05:45.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:44 smithi102 ceph-mon[41371]: from='client.47980 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36"}]': finished 2024-09-11T08:05:45.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:44 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:05:45.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:44 smithi102 ceph-mon[41371]: osdmap e216: 8 total, 8 up, 8 in 2024-09-11T08:05:45.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:44 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4177807146' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:45.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:44 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-27"}]: dispatch 2024-09-11T08:05:45.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:44 smithi102 ceph-mon[41371]: from='client.47980 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:45.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:44 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-27"}]: dispatch 2024-09-11T08:05:45.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:44 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:05:45.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:44 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:05:45.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:44 smithi102 ceph-mon[41371]: pgmap v293: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:05:45.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:44 smithi102 ceph-mon[29849]: from='client.47980 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36"}]': finished 2024-09-11T08:05:45.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:44 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:05:45.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:44 smithi102 ceph-mon[29849]: osdmap e216: 8 total, 8 up, 8 in 2024-09-11T08:05:45.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:44 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4177807146' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:45.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:44 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-27"}]: dispatch 2024-09-11T08:05:45.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:44 smithi102 ceph-mon[29849]: from='client.47980 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:45.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:44 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-27"}]: dispatch 2024-09-11T08:05:45.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:44 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:05:45.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:44 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:05:45.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:44 smithi102 ceph-mon[29849]: pgmap v293: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:05:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:44 smithi146 ceph-mon[33158]: from='client.47980 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi102-80338-36"}]': finished 2024-09-11T08:05:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:44 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:05:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:44 smithi146 ceph-mon[33158]: osdmap e216: 8 total, 8 up, 8 in 2024-09-11T08:05:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:44 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4177807146' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:45.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:44 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-27"}]: dispatch 2024-09-11T08:05:45.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:44 smithi146 ceph-mon[33158]: from='client.47980 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi102-80338-36"}]: dispatch 2024-09-11T08:05:45.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:44 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-27"}]: dispatch 2024-09-11T08:05:45.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:44 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:05:45.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:44 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:05:45.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:44 smithi146 ceph-mon[33158]: pgmap v293: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:05:46.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:46 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:05:46.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:46 smithi146 ceph-mon[33158]: from='client.46889 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi102-80356-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi102-80356-46"}]': finished 2024-09-11T08:05:46.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:46 smithi146 ceph-mon[33158]: from='client.47980 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi102-80338-36"}]': finished 2024-09-11T08:05:46.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:46 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-27"}]': finished 2024-09-11T08:05:46.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:46 smithi146 ceph-mon[33158]: osdmap e217: 8 total, 8 up, 8 in 2024-09-11T08:05:46.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:46 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi102-80338-37"}]: dispatch 2024-09-11T08:05:46.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:46 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi102-80338-37"}]: dispatch 2024-09-11T08:05:46.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:46 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi102-80338-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:46.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:46 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi102-80338-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:46.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:46 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:05:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:46 smithi102 ceph-mon[41371]: from='client.46889 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi102-80356-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi102-80356-46"}]': finished 2024-09-11T08:05:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:46 smithi102 ceph-mon[41371]: from='client.47980 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi102-80338-36"}]': finished 2024-09-11T08:05:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:46 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-27"}]': finished 2024-09-11T08:05:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:46 smithi102 ceph-mon[41371]: osdmap e217: 8 total, 8 up, 8 in 2024-09-11T08:05:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:46 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi102-80338-37"}]: dispatch 2024-09-11T08:05:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:46 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi102-80338-37"}]: dispatch 2024-09-11T08:05:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:46 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi102-80338-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:46 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi102-80338-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:46.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:46 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:05:46.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:46 smithi102 ceph-mon[29849]: from='client.46889 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi102-80356-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi102-80356-46"}]': finished 2024-09-11T08:05:46.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:46 smithi102 ceph-mon[29849]: from='client.47980 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi102-80338-36"}]': finished 2024-09-11T08:05:46.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:46 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-27"}]': finished 2024-09-11T08:05:46.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:46 smithi102 ceph-mon[29849]: osdmap e217: 8 total, 8 up, 8 in 2024-09-11T08:05:46.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi102-80338-37"}]: dispatch 2024-09-11T08:05:46.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi102-80338-37"}]: dispatch 2024-09-11T08:05:46.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi102-80338-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:46.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi102-80338-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:47.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:47 smithi102 ceph-mon[29849]: osdmap e218: 8 total, 8 up, 8 in 2024-09-11T08:05:47.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi102-80338-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi102-80338-37"}]: dispatch 2024-09-11T08:05:47.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:47 smithi102 ceph-mon[29849]: pgmap v296: 204 pgs: 8 creating+peering, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:05:47.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:47 smithi102 ceph-mon[29849]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:47.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:47 smithi102 ceph-mon[41371]: osdmap e218: 8 total, 8 up, 8 in 2024-09-11T08:05:47.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi102-80338-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi102-80338-37"}]: dispatch 2024-09-11T08:05:47.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:47 smithi102 ceph-mon[41371]: pgmap v296: 204 pgs: 8 creating+peering, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:05:47.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:47 smithi102 ceph-mon[41371]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:47.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:47 smithi146 ceph-mon[33158]: osdmap e218: 8 total, 8 up, 8 in 2024-09-11T08:05:47.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi102-80338-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi102-80338-37"}]: dispatch 2024-09-11T08:05:47.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:47 smithi146 ceph-mon[33158]: pgmap v296: 204 pgs: 8 creating+peering, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:05:47.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:47 smithi146 ceph-mon[33158]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:48.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:48 smithi102 ceph-mon[41371]: osdmap e219: 8 total, 8 up, 8 in 2024-09-11T08:05:48.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1738569373' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:48.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:48.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:48 smithi102 ceph-mon[41371]: from='client.46889 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:48.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:48 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:48.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi102-80338-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi102-80338-37"}]': finished 2024-09-11T08:05:48.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:48 smithi102 ceph-mon[41371]: from='client.46889 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-46"}]': finished 2024-09-11T08:05:48.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:48 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:48.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:48 smithi102 ceph-mon[41371]: osdmap e220: 8 total, 8 up, 8 in 2024-09-11T08:05:48.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1738569373' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:48.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:48 smithi102 ceph-mon[41371]: from='client.46889 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:48.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:48 smithi102 ceph-mon[29849]: osdmap e219: 8 total, 8 up, 8 in 2024-09-11T08:05:48.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1738569373' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:48.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:48.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:48 smithi102 ceph-mon[29849]: from='client.46889 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:48.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:48 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:48.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi102-80338-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi102-80338-37"}]': finished 2024-09-11T08:05:48.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:48 smithi102 ceph-mon[29849]: from='client.46889 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-46"}]': finished 2024-09-11T08:05:48.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:48 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:48.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:48 smithi102 ceph-mon[29849]: osdmap e220: 8 total, 8 up, 8 in 2024-09-11T08:05:48.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1738569373' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:48.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:48 smithi102 ceph-mon[29849]: from='client.46889 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:48.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:48 smithi146 ceph-mon[33158]: osdmap e219: 8 total, 8 up, 8 in 2024-09-11T08:05:48.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1738569373' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:48.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:48.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:48 smithi146 ceph-mon[33158]: from='client.46889 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:48.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:48 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:05:48.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi102-80338-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi102-80338-37"}]': finished 2024-09-11T08:05:48.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:48 smithi146 ceph-mon[33158]: from='client.46889 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-46"}]': finished 2024-09-11T08:05:48.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:48 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:05:48.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:48 smithi146 ceph-mon[33158]: osdmap e220: 8 total, 8 up, 8 in 2024-09-11T08:05:48.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1738569373' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:48.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:48 smithi146 ceph-mon[33158]: from='client.46889 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-46"}]: dispatch 2024-09-11T08:05:49.573 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:49 smithi146 ceph-mon[33158]: pgmap v299: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:05:49.573 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:49 smithi146 ceph-mon[33158]: from='client.46889 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-46"}]': finished 2024-09-11T08:05:49.574 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:49 smithi146 ceph-mon[33158]: osdmap e221: 8 total, 8 up, 8 in 2024-09-11T08:05:49.574 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:49 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1823428028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:49.574 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:49 smithi146 ceph-mon[33158]: from='client.46901 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:49.574 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:49 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1823428028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:49.574 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:49 smithi146 ceph-mon[33158]: from='client.46901 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:49.574 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:49 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1823428028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi102-80356-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:49.574 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:49 smithi146 ceph-mon[33158]: from='client.46901 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi102-80356-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:49.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:49 smithi102 ceph-mon[41371]: pgmap v299: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:05:49.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:49 smithi102 ceph-mon[41371]: from='client.46889 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-46"}]': finished 2024-09-11T08:05:49.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:49 smithi102 ceph-mon[41371]: osdmap e221: 8 total, 8 up, 8 in 2024-09-11T08:05:49.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:49 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1823428028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:49.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:49 smithi102 ceph-mon[41371]: from='client.46901 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:49.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:49 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1823428028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:49.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:49 smithi102 ceph-mon[41371]: from='client.46901 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:49.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:49 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1823428028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi102-80356-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:49.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:49 smithi102 ceph-mon[41371]: from='client.46901 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi102-80356-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:49.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:49 smithi102 ceph-mon[29849]: pgmap v299: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:05:49.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:49 smithi102 ceph-mon[29849]: from='client.46889 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-46"}]': finished 2024-09-11T08:05:49.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:49 smithi102 ceph-mon[29849]: osdmap e221: 8 total, 8 up, 8 in 2024-09-11T08:05:49.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:49 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1823428028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:49.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:49 smithi102 ceph-mon[29849]: from='client.46901 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:49.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:49 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1823428028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:49.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:49 smithi102 ceph-mon[29849]: from='client.46901 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:49.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:49 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1823428028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi102-80356-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:49.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:49 smithi102 ceph-mon[29849]: from='client.46901 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi102-80356-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:49.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:05:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:05:50.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:05:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:05:49] "GET /metrics HTTP/1.1" 200 228466 "" "Prometheus/2.43.0" 2024-09-11T08:05:50.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:50 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:05:50.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:50 smithi102 ceph-mon[41371]: from='client.46901 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi102-80356-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:50.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:50 smithi102 ceph-mon[41371]: osdmap e222: 8 total, 8 up, 8 in 2024-09-11T08:05:50.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:50 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi102-80338-37"}]: dispatch 2024-09-11T08:05:50.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:50 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1823428028' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi102-80356-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:50.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:50 smithi102 ceph-mon[41371]: from='client.46901 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi102-80356-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:50.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:50 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:50.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:50 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:50.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:50 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:05:50.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:50 smithi102 ceph-mon[29849]: from='client.46901 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi102-80356-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:50.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:50 smithi102 ceph-mon[29849]: osdmap e222: 8 total, 8 up, 8 in 2024-09-11T08:05:50.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:50 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi102-80338-37"}]: dispatch 2024-09-11T08:05:50.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:50 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1823428028' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi102-80356-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:50.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:50 smithi102 ceph-mon[29849]: from='client.46901 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi102-80356-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:50.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:50 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:50.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:50 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:50.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:50 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:05:50.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:50 smithi146 ceph-mon[33158]: from='client.46901 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi102-80356-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:50.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:50 smithi146 ceph-mon[33158]: osdmap e222: 8 total, 8 up, 8 in 2024-09-11T08:05:50.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:50 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi102-80338-37"}]: dispatch 2024-09-11T08:05:50.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:50 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1823428028' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi102-80356-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:50.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:50 smithi146 ceph-mon[33158]: from='client.46901 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi102-80356-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:50.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:50 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:50.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:50 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:05:51.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:51 smithi102 ceph-mon[29849]: pgmap v302: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-09-11T08:05:51.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:51 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi102-80338-37"}]': finished 2024-09-11T08:05:51.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:51 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:05:51.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:51 smithi102 ceph-mon[29849]: osdmap e223: 8 total, 8 up, 8 in 2024-09-11T08:05:51.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:51 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi102-80338-37"}]: dispatch 2024-09-11T08:05:51.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:51 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-29"}]: dispatch 2024-09-11T08:05:51.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:51 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-29"}]: dispatch 2024-09-11T08:05:51.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:51 smithi102 ceph-mon[41371]: pgmap v302: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-09-11T08:05:51.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:51 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi102-80338-37"}]': finished 2024-09-11T08:05:51.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:51 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:05:51.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:51 smithi102 ceph-mon[41371]: osdmap e223: 8 total, 8 up, 8 in 2024-09-11T08:05:51.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:51 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi102-80338-37"}]: dispatch 2024-09-11T08:05:51.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:51 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-29"}]: dispatch 2024-09-11T08:05:51.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:51 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-29"}]: dispatch 2024-09-11T08:05:51.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:51 smithi146 ceph-mon[33158]: pgmap v302: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-09-11T08:05:51.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:51 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi102-80338-37"}]': finished 2024-09-11T08:05:51.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:51 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:05:51.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:51 smithi146 ceph-mon[33158]: osdmap e223: 8 total, 8 up, 8 in 2024-09-11T08:05:51.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:51 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi102-80338-37"}]: dispatch 2024-09-11T08:05:51.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:51 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-29"}]: dispatch 2024-09-11T08:05:51.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:51 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-29"}]: dispatch 2024-09-11T08:05:53.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:52 smithi146 ceph-mon[33158]: from='client.46901 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi102-80356-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi102-80356-47"}]': finished 2024-09-11T08:05:53.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:52 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi102-80338-37"}]': finished 2024-09-11T08:05:53.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:52 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-29"}]': finished 2024-09-11T08:05:53.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:52 smithi146 ceph-mon[33158]: osdmap e224: 8 total, 8 up, 8 in 2024-09-11T08:05:53.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:52 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-29", "mode": "writeback"}]: dispatch 2024-09-11T08:05:53.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:52 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-29", "mode": "writeback"}]: dispatch 2024-09-11T08:05:53.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:52 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/894741037' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:53.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:52 smithi146 ceph-mon[33158]: from='client.46907 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:53.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:52 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/894741037' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:53.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:52 smithi146 ceph-mon[33158]: from='client.46907 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:53.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:52 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/894741037' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi102-80338-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:53.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:52 smithi146 ceph-mon[33158]: from='client.46907 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi102-80338-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:53.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[41371]: from='client.46901 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi102-80356-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi102-80356-47"}]': finished 2024-09-11T08:05:53.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi102-80338-37"}]': finished 2024-09-11T08:05:53.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-29"}]': finished 2024-09-11T08:05:53.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[41371]: osdmap e224: 8 total, 8 up, 8 in 2024-09-11T08:05:53.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-29", "mode": "writeback"}]: dispatch 2024-09-11T08:05:53.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-29", "mode": "writeback"}]: dispatch 2024-09-11T08:05:53.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/894741037' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:53.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[41371]: from='client.46907 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:53.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/894741037' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:53.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[41371]: from='client.46907 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:53.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/894741037' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi102-80338-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:53.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[41371]: from='client.46907 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi102-80338-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:53.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[29849]: from='client.46901 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi102-80356-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi102-80356-47"}]': finished 2024-09-11T08:05:53.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3579777507' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi102-80338-37"}]': finished 2024-09-11T08:05:53.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-29"}]': finished 2024-09-11T08:05:53.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[29849]: osdmap e224: 8 total, 8 up, 8 in 2024-09-11T08:05:53.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-29", "mode": "writeback"}]: dispatch 2024-09-11T08:05:53.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-29", "mode": "writeback"}]: dispatch 2024-09-11T08:05:53.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/894741037' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:53.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[29849]: from='client.46907 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:53.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/894741037' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:53.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[29849]: from='client.46907 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:53.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/894741037' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi102-80338-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:53.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:52 smithi102 ceph-mon[29849]: from='client.46907 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi102-80338-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:54.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:54 smithi102 ceph-mon[41371]: pgmap v305: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-11T08:05:54.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:54 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:54.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:54 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:05:54.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:54 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-29", "mode": "writeback"}]': finished 2024-09-11T08:05:54.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:54 smithi102 ceph-mon[41371]: from='client.46907 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi102-80338-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:54.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:54 smithi102 ceph-mon[41371]: osdmap e225: 8 total, 8 up, 8 in 2024-09-11T08:05:54.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:54 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/894741037' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi102-80338-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:54.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:54 smithi102 ceph-mon[41371]: from='client.46907 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi102-80338-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:54.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:54 smithi102 ceph-mon[29849]: pgmap v305: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-11T08:05:54.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:54 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:54.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:54 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:05:54.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:54 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-29", "mode": "writeback"}]': finished 2024-09-11T08:05:54.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:54 smithi102 ceph-mon[29849]: from='client.46907 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi102-80338-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:54.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:54 smithi102 ceph-mon[29849]: osdmap e225: 8 total, 8 up, 8 in 2024-09-11T08:05:54.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:54 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/894741037' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi102-80338-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:54.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:54 smithi102 ceph-mon[29849]: from='client.46907 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi102-80338-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:54.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:54 smithi146 ceph-mon[33158]: pgmap v305: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-11T08:05:54.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:54 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:05:54.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:54 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:05:54.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:54 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-29", "mode": "writeback"}]': finished 2024-09-11T08:05:54.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:54 smithi146 ceph-mon[33158]: from='client.46907 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi102-80338-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:54.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:54 smithi146 ceph-mon[33158]: osdmap e225: 8 total, 8 up, 8 in 2024-09-11T08:05:54.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:54 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/894741037' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi102-80338-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:54.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:54 smithi146 ceph-mon[33158]: from='client.46907 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi102-80338-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:54.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:05:54 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:05:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:05:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:55 smithi102 ceph-mon[29849]: osdmap e226: 8 total, 8 up, 8 in 2024-09-11T08:05:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1823428028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:55 smithi102 ceph-mon[29849]: from='client.46901 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:55 smithi102 ceph-mon[29849]: pgmap v308: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:05:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:55 smithi102 ceph-mon[29849]: from='client.46907 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi102-80338-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi102-80338-38"}]': finished 2024-09-11T08:05:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:55 smithi102 ceph-mon[29849]: from='client.46901 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-47"}]': finished 2024-09-11T08:05:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:55 smithi102 ceph-mon[29849]: osdmap e227: 8 total, 8 up, 8 in 2024-09-11T08:05:55.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1823428028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:55.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:55 smithi102 ceph-mon[29849]: from='client.46901 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:55.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:55 smithi102 ceph-mon[41371]: osdmap e226: 8 total, 8 up, 8 in 2024-09-11T08:05:55.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1823428028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:55.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:55 smithi102 ceph-mon[41371]: from='client.46901 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:55.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:55 smithi102 ceph-mon[41371]: pgmap v308: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:05:55.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:55 smithi102 ceph-mon[41371]: from='client.46907 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi102-80338-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi102-80338-38"}]': finished 2024-09-11T08:05:55.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:55 smithi102 ceph-mon[41371]: from='client.46901 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-47"}]': finished 2024-09-11T08:05:55.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:55 smithi102 ceph-mon[41371]: osdmap e227: 8 total, 8 up, 8 in 2024-09-11T08:05:55.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1823428028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:55.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:55 smithi102 ceph-mon[41371]: from='client.46901 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:55 smithi146 ceph-mon[33158]: osdmap e226: 8 total, 8 up, 8 in 2024-09-11T08:05:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1823428028' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:55 smithi146 ceph-mon[33158]: from='client.46901 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:55 smithi146 ceph-mon[33158]: pgmap v308: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:05:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:55 smithi146 ceph-mon[33158]: from='client.46907 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi102-80338-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi102-80338-38"}]': finished 2024-09-11T08:05:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:55 smithi146 ceph-mon[33158]: from='client.46901 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi102-80356-47"}]': finished 2024-09-11T08:05:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:55 smithi146 ceph-mon[33158]: osdmap e227: 8 total, 8 up, 8 in 2024-09-11T08:05:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1823428028' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:55 smithi146 ceph-mon[33158]: from='client.46901 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-47"}]: dispatch 2024-09-11T08:05:57.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:56 smithi146 ceph-mon[33158]: from='client.46901 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-47"}]': finished 2024-09-11T08:05:57.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:56 smithi146 ceph-mon[33158]: osdmap e228: 8 total, 8 up, 8 in 2024-09-11T08:05:57.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:56 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/586876656' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:05:57.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:56 smithi146 ceph-mon[33158]: from='client.46913 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:05:57.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:56 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/586876656' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:05:57.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:56 smithi146 ceph-mon[33158]: from='client.46913 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:05:57.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:56 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/586876656' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi102-80356-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:57.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:56 smithi146 ceph-mon[33158]: from='client.46913 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi102-80356-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:57.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:56 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:57.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:56 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:56 smithi102 ceph-mon[41371]: from='client.46901 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-47"}]': finished 2024-09-11T08:05:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:56 smithi102 ceph-mon[41371]: osdmap e228: 8 total, 8 up, 8 in 2024-09-11T08:05:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:56 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/586876656' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:05:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:56 smithi102 ceph-mon[41371]: from='client.46913 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:05:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:56 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/586876656' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:05:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:56 smithi102 ceph-mon[41371]: from='client.46913 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:05:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:56 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/586876656' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi102-80356-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:56 smithi102 ceph-mon[41371]: from='client.46913 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi102-80356-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:57.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:56 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:57.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:56 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:57.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:56 smithi102 ceph-mon[29849]: from='client.46901 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi102-80356-47"}]': finished 2024-09-11T08:05:57.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:56 smithi102 ceph-mon[29849]: osdmap e228: 8 total, 8 up, 8 in 2024-09-11T08:05:57.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:56 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/586876656' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:05:57.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:56 smithi102 ceph-mon[29849]: from='client.46913 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:05:57.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:56 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/586876656' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:05:57.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:56 smithi102 ceph-mon[29849]: from='client.46913 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:05:57.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:56 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/586876656' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi102-80356-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:57.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:56 smithi102 ceph-mon[29849]: from='client.46913 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi102-80356-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:05:57.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:56 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:57.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:56 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:05:58.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:58 smithi146 ceph-mon[33158]: pgmap v311: 236 pgs: 8 creating+peering, 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-11T08:05:58.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:58 smithi146 ceph-mon[33158]: from='client.46913 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi102-80356-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:58.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:58 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:05:58.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:58 smithi146 ceph-mon[33158]: osdmap e229: 8 total, 8 up, 8 in 2024-09-11T08:05:58.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:58 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-29"}]: dispatch 2024-09-11T08:05:58.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:58 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-29"}]: dispatch 2024-09-11T08:05:58.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:58 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/894741037' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:58.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:58 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/586876656' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi102-80356-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:05:58.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:58 smithi146 ceph-mon[33158]: from='client.46907 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:58.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:58 smithi146 ceph-mon[33158]: from='client.46913 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi102-80356-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:05:58.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:58 smithi102 ceph-mon[41371]: pgmap v311: 236 pgs: 8 creating+peering, 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-11T08:05:58.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:58 smithi102 ceph-mon[41371]: from='client.46913 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi102-80356-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:58.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:58 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:05:58.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:58 smithi102 ceph-mon[41371]: osdmap e229: 8 total, 8 up, 8 in 2024-09-11T08:05:58.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:58 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-29"}]: dispatch 2024-09-11T08:05:58.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:58 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-29"}]: dispatch 2024-09-11T08:05:58.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:58 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/894741037' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:58.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:58 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/586876656' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi102-80356-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:05:58.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:58 smithi102 ceph-mon[41371]: from='client.46907 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:58.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:58 smithi102 ceph-mon[41371]: from='client.46913 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi102-80356-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:05:58.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:58 smithi102 ceph-mon[29849]: pgmap v311: 236 pgs: 8 creating+peering, 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-11T08:05:58.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:58 smithi102 ceph-mon[29849]: from='client.46913 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi102-80356-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:05:58.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:58 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:05:58.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:58 smithi102 ceph-mon[29849]: osdmap e229: 8 total, 8 up, 8 in 2024-09-11T08:05:58.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:58 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-29"}]: dispatch 2024-09-11T08:05:58.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:58 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-29"}]: dispatch 2024-09-11T08:05:58.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:58 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/894741037' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:58.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:58 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/586876656' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi102-80356-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:05:58.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:58 smithi102 ceph-mon[29849]: from='client.46907 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:58.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:58 smithi102 ceph-mon[29849]: from='client.46913 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi102-80356-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:05:59.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:59 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:05:59.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:59 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-29"}]': finished 2024-09-11T08:05:59.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:59 smithi146 ceph-mon[33158]: from='client.46907 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi102-80338-38"}]': finished 2024-09-11T08:05:59.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:59 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/894741037' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:59.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:59 smithi146 ceph-mon[33158]: osdmap e230: 8 total, 8 up, 8 in 2024-09-11T08:05:59.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:59 smithi146 ceph-mon[33158]: from='client.46907 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:59.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:59 smithi146 ceph-mon[33158]: pgmap v314: 228 pgs: 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-11T08:05:59.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:59 smithi146 ceph-mon[33158]: from='client.46913 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi102-80356-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi102-80356-48"}]': finished 2024-09-11T08:05:59.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:59 smithi146 ceph-mon[33158]: from='client.46907 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi102-80338-38"}]': finished 2024-09-11T08:05:59.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:59 smithi146 ceph-mon[33158]: osdmap e231: 8 total, 8 up, 8 in 2024-09-11T08:05:59.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:05:59 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2879938457' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:05:59.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:59 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:05:59.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:59 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-29"}]': finished 2024-09-11T08:05:59.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:59 smithi102 ceph-mon[29849]: from='client.46907 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi102-80338-38"}]': finished 2024-09-11T08:05:59.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:59 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/894741037' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:59.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:59 smithi102 ceph-mon[29849]: osdmap e230: 8 total, 8 up, 8 in 2024-09-11T08:05:59.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:59 smithi102 ceph-mon[29849]: from='client.46907 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:59.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:59 smithi102 ceph-mon[29849]: pgmap v314: 228 pgs: 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-11T08:05:59.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:59 smithi102 ceph-mon[29849]: from='client.46913 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi102-80356-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi102-80356-48"}]': finished 2024-09-11T08:05:59.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:59 smithi102 ceph-mon[29849]: from='client.46907 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi102-80338-38"}]': finished 2024-09-11T08:05:59.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:59 smithi102 ceph-mon[29849]: osdmap e231: 8 total, 8 up, 8 in 2024-09-11T08:05:59.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:05:59 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2879938457' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:05:59.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:59 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:05:59.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:59 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-29"}]': finished 2024-09-11T08:05:59.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:59 smithi102 ceph-mon[41371]: from='client.46907 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi102-80338-38"}]': finished 2024-09-11T08:05:59.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:59 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/894741037' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:59.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:59 smithi102 ceph-mon[41371]: osdmap e230: 8 total, 8 up, 8 in 2024-09-11T08:05:59.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:59 smithi102 ceph-mon[41371]: from='client.46907 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi102-80338-38"}]: dispatch 2024-09-11T08:05:59.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:59 smithi102 ceph-mon[41371]: pgmap v314: 228 pgs: 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-11T08:05:59.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:59 smithi102 ceph-mon[41371]: from='client.46913 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi102-80356-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi102-80356-48"}]': finished 2024-09-11T08:05:59.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:59 smithi102 ceph-mon[41371]: from='client.46907 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi102-80338-38"}]': finished 2024-09-11T08:05:59.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:59 smithi102 ceph-mon[41371]: osdmap e231: 8 total, 8 up, 8 in 2024-09-11T08:05:59.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:05:59 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2879938457' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:05:59.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:05:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:06:00.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:05:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:05:59] "GET /metrics HTTP/1.1" 200 228493 "" "Prometheus/2.43.0" 2024-09-11T08:06:00.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:00 smithi146 ceph-mon[33158]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:00.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:00 smithi146 ceph-mon[33158]: from='client.46919 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:00.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:00 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2879938457' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:00.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:00 smithi146 ceph-mon[33158]: from='client.46919 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:00.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:00 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2879938457' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi102-80338-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:00.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:00 smithi146 ceph-mon[33158]: from='client.46919 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi102-80338-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:00.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:00 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:06:00.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:00 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:06:00.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:00 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:06:00.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:00 smithi146 ceph-mon[33158]: from='client.46919 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi102-80338-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:00.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:00 smithi146 ceph-mon[33158]: osdmap e232: 8 total, 8 up, 8 in 2024-09-11T08:06:00.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:00 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:00.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:00 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:00.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:00 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2879938457' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi102-80338-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:00.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:00 smithi146 ceph-mon[33158]: from='client.46919 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi102-80338-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[41371]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[41371]: from='client.46919 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2879938457' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[41371]: from='client.46919 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2879938457' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi102-80338-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[41371]: from='client.46919 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi102-80338-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:06:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:06:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:06:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[41371]: from='client.46919 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi102-80338-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:00.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[41371]: osdmap e232: 8 total, 8 up, 8 in 2024-09-11T08:06:00.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:00.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:00.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2879938457' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi102-80338-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:00.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[41371]: from='client.46919 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi102-80338-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:00.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[29849]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:00.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[29849]: from='client.46919 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:00.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2879938457' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:00.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[29849]: from='client.46919 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:00.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2879938457' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi102-80338-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:00.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[29849]: from='client.46919 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi102-80338-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:00.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:06:00.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:06:00.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:06:00.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[29849]: from='client.46919 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi102-80338-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:00.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[29849]: osdmap e232: 8 total, 8 up, 8 in 2024-09-11T08:06:00.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:00.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:00.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2879938457' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi102-80338-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:00.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:00 smithi102 ceph-mon[29849]: from='client.46919 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi102-80338-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:02.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:01 smithi146 ceph-mon[33158]: pgmap v317: 236 pgs: 32 creating+peering, 204 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:02.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:01 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:02.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:01 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:02.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:01 smithi146 ceph-mon[33158]: osdmap e233: 8 total, 8 up, 8 in 2024-09-11T08:06:02.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:01 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:02.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:01 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/586876656' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:06:02.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:01 smithi146 ceph-mon[33158]: from='client.46913 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:06:02.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:01 smithi146 ceph-mon[33158]: from='client.46919 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi102-80338-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi102-80338-39"}]': finished 2024-09-11T08:06:02.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:01 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:06:02.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:01 smithi146 ceph-mon[33158]: from='client.46913 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi102-80356-48"}]': finished 2024-09-11T08:06:02.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:01 smithi146 ceph-mon[33158]: osdmap e234: 8 total, 8 up, 8 in 2024-09-11T08:06:02.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:01 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-31"}]: dispatch 2024-09-11T08:06:02.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:01 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-31"}]: dispatch 2024-09-11T08:06:02.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:01 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/586876656' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:06:02.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:01 smithi146 ceph-mon[33158]: from='client.46913 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:06:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[29849]: pgmap v317: 236 pgs: 32 creating+peering, 204 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[29849]: osdmap e233: 8 total, 8 up, 8 in 2024-09-11T08:06:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/586876656' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:06:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[29849]: from='client.46913 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:06:02.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[29849]: from='client.46919 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi102-80338-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi102-80338-39"}]': finished 2024-09-11T08:06:02.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:06:02.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[29849]: from='client.46913 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi102-80356-48"}]': finished 2024-09-11T08:06:02.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[29849]: osdmap e234: 8 total, 8 up, 8 in 2024-09-11T08:06:02.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-31"}]: dispatch 2024-09-11T08:06:02.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-31"}]: dispatch 2024-09-11T08:06:02.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/586876656' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:06:02.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[29849]: from='client.46913 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:06:02.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[41371]: pgmap v317: 236 pgs: 32 creating+peering, 204 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:02.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:02.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:02.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[41371]: osdmap e233: 8 total, 8 up, 8 in 2024-09-11T08:06:02.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:02.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/586876656' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:06:02.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[41371]: from='client.46913 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:06:02.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[41371]: from='client.46919 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi102-80338-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi102-80338-39"}]': finished 2024-09-11T08:06:02.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:06:02.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[41371]: from='client.46913 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi102-80356-48"}]': finished 2024-09-11T08:06:02.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[41371]: osdmap e234: 8 total, 8 up, 8 in 2024-09-11T08:06:02.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-31"}]: dispatch 2024-09-11T08:06:02.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-31"}]: dispatch 2024-09-11T08:06:02.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/586876656' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:06:02.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:01 smithi102 ceph-mon[41371]: from='client.46913 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi102-80356-48"}]: dispatch 2024-09-11T08:06:04.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:03 smithi146 ceph-mon[33158]: pgmap v320: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:04.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:03 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-31"}]': finished 2024-09-11T08:06:04.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:03 smithi146 ceph-mon[33158]: from='client.46913 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi102-80356-48"}]': finished 2024-09-11T08:06:04.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:03 smithi146 ceph-mon[33158]: osdmap e235: 8 total, 8 up, 8 in 2024-09-11T08:06:04.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:03 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-31", "mode": "writeback"}]: dispatch 2024-09-11T08:06:04.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:03 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-31", "mode": "writeback"}]: dispatch 2024-09-11T08:06:04.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:03 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3777328099' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:04.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:03 smithi146 ceph-mon[33158]: from='client.46922 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:04.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:03 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3777328099' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:04.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:03 smithi146 ceph-mon[33158]: from='client.46922 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:04.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:03 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3777328099' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi102-80356-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:04.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:03 smithi146 ceph-mon[33158]: from='client.46922 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi102-80356-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:04.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[29849]: pgmap v320: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:04.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-31"}]': finished 2024-09-11T08:06:04.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[29849]: from='client.46913 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi102-80356-48"}]': finished 2024-09-11T08:06:04.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[29849]: osdmap e235: 8 total, 8 up, 8 in 2024-09-11T08:06:04.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-31", "mode": "writeback"}]: dispatch 2024-09-11T08:06:04.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-31", "mode": "writeback"}]: dispatch 2024-09-11T08:06:04.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3777328099' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:04.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[29849]: from='client.46922 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:04.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3777328099' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:04.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[29849]: from='client.46922 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:04.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3777328099' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi102-80356-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:04.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[29849]: from='client.46922 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi102-80356-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:04.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[41371]: pgmap v320: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:04.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-31"}]': finished 2024-09-11T08:06:04.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[41371]: from='client.46913 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi102-80356-48"}]': finished 2024-09-11T08:06:04.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[41371]: osdmap e235: 8 total, 8 up, 8 in 2024-09-11T08:06:04.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-31", "mode": "writeback"}]: dispatch 2024-09-11T08:06:04.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-31", "mode": "writeback"}]: dispatch 2024-09-11T08:06:04.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3777328099' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:04.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[41371]: from='client.46922 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:04.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3777328099' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:04.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[41371]: from='client.46922 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:04.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3777328099' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi102-80356-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:04.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:03 smithi102 ceph-mon[41371]: from='client.46922 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi102-80356-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:04.911 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:06:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:06:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:06:05.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:04 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:06:05.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:04 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-31", "mode": "writeback"}]': finished 2024-09-11T08:06:05.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:04 smithi146 ceph-mon[33158]: from='client.46922 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi102-80356-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:05.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:04 smithi146 ceph-mon[33158]: osdmap e236: 8 total, 8 up, 8 in 2024-09-11T08:06:05.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:04 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3777328099' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi102-80356-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:05.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:04 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2879938457' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:05.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:04 smithi146 ceph-mon[33158]: from='client.46922 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi102-80356-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:05.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:04 smithi146 ceph-mon[33158]: from='client.46919 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:05.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:04 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:05.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:04 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:04 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:06:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:04 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-31", "mode": "writeback"}]': finished 2024-09-11T08:06:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:04 smithi102 ceph-mon[29849]: from='client.46922 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi102-80356-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:04 smithi102 ceph-mon[29849]: osdmap e236: 8 total, 8 up, 8 in 2024-09-11T08:06:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:04 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3777328099' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi102-80356-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:04 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2879938457' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:04 smithi102 ceph-mon[29849]: from='client.46922 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi102-80356-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:04 smithi102 ceph-mon[29849]: from='client.46919 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:04 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:04 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:04 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:06:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:04 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-31", "mode": "writeback"}]': finished 2024-09-11T08:06:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:04 smithi102 ceph-mon[41371]: from='client.46922 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi102-80356-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:04 smithi102 ceph-mon[41371]: osdmap e236: 8 total, 8 up, 8 in 2024-09-11T08:06:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:04 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3777328099' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi102-80356-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:04 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2879938457' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:04 smithi102 ceph-mon[41371]: from='client.46922 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi102-80356-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:04 smithi102 ceph-mon[41371]: from='client.46919 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:04 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:04 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:06.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:05 smithi146 ceph-mon[33158]: pgmap v323: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:06.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:05 smithi146 ceph-mon[33158]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:06.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:05 smithi146 ceph-mon[33158]: from='client.46919 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi102-80338-39"}]': finished 2024-09-11T08:06:06.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:05 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:06:06.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:05 smithi146 ceph-mon[33158]: osdmap e237: 8 total, 8 up, 8 in 2024-09-11T08:06:06.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:05 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2879938457' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:06.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:05 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-31"}]: dispatch 2024-09-11T08:06:06.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:05 smithi146 ceph-mon[33158]: from='client.46919 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:06.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:05 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-31"}]: dispatch 2024-09-11T08:06:06.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:05 smithi102 ceph-mon[29849]: pgmap v323: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:06.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:05 smithi102 ceph-mon[29849]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:06.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:05 smithi102 ceph-mon[29849]: from='client.46919 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi102-80338-39"}]': finished 2024-09-11T08:06:06.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:05 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:06:06.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:05 smithi102 ceph-mon[29849]: osdmap e237: 8 total, 8 up, 8 in 2024-09-11T08:06:06.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:05 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2879938457' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:06.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:05 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-31"}]: dispatch 2024-09-11T08:06:06.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:05 smithi102 ceph-mon[29849]: from='client.46919 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:06.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:05 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-31"}]: dispatch 2024-09-11T08:06:06.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:05 smithi102 ceph-mon[41371]: pgmap v323: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:06.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:05 smithi102 ceph-mon[41371]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:06.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:05 smithi102 ceph-mon[41371]: from='client.46919 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi102-80338-39"}]': finished 2024-09-11T08:06:06.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:05 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:06:06.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:05 smithi102 ceph-mon[41371]: osdmap e237: 8 total, 8 up, 8 in 2024-09-11T08:06:06.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:05 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2879938457' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:06.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:05 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-31"}]: dispatch 2024-09-11T08:06:06.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:05 smithi102 ceph-mon[41371]: from='client.46919 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi102-80338-39"}]: dispatch 2024-09-11T08:06:06.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:05 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-31"}]: dispatch 2024-09-11T08:06:07.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:06 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:06:07.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:06 smithi146 ceph-mon[33158]: from='client.46922 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi102-80356-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi102-80356-49"}]': finished 2024-09-11T08:06:07.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:06 smithi146 ceph-mon[33158]: from='client.46919 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi102-80338-39"}]': finished 2024-09-11T08:06:07.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:06 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-31"}]': finished 2024-09-11T08:06:07.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:06 smithi146 ceph-mon[33158]: osdmap e238: 8 total, 8 up, 8 in 2024-09-11T08:06:07.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:06 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1678854715' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:07.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:06 smithi146 ceph-mon[33158]: from='client.46928 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:07.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:06 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1678854715' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:07.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:06 smithi146 ceph-mon[33158]: from='client.46928 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:07.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:06 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1678854715' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi102-80338-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:07.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:06 smithi146 ceph-mon[33158]: from='client.46928 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi102-80338-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:07.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:06 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:06:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:06 smithi102 ceph-mon[29849]: from='client.46922 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi102-80356-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi102-80356-49"}]': finished 2024-09-11T08:06:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:06 smithi102 ceph-mon[29849]: from='client.46919 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi102-80338-39"}]': finished 2024-09-11T08:06:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:06 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-31"}]': finished 2024-09-11T08:06:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:06 smithi102 ceph-mon[29849]: osdmap e238: 8 total, 8 up, 8 in 2024-09-11T08:06:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:06 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1678854715' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:06 smithi102 ceph-mon[29849]: from='client.46928 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:06 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1678854715' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:06 smithi102 ceph-mon[29849]: from='client.46928 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:06 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1678854715' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi102-80338-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:06 smithi102 ceph-mon[29849]: from='client.46928 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi102-80338-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:07.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:06 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:06:07.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:06 smithi102 ceph-mon[41371]: from='client.46922 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi102-80356-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi102-80356-49"}]': finished 2024-09-11T08:06:07.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:06 smithi102 ceph-mon[41371]: from='client.46919 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi102-80338-39"}]': finished 2024-09-11T08:06:07.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:06 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-31"}]': finished 2024-09-11T08:06:07.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:06 smithi102 ceph-mon[41371]: osdmap e238: 8 total, 8 up, 8 in 2024-09-11T08:06:07.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:06 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1678854715' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:07.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:06 smithi102 ceph-mon[41371]: from='client.46928 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:07.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:06 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1678854715' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:07.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:06 smithi102 ceph-mon[41371]: from='client.46928 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:07.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:06 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1678854715' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi102-80338-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:07.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:06 smithi102 ceph-mon[41371]: from='client.46928 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi102-80338-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:08.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:07 smithi146 ceph-mon[33158]: pgmap v326: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:06:08.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:07 smithi146 ceph-mon[33158]: from='client.46928 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi102-80338-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:08.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:07 smithi146 ceph-mon[33158]: osdmap e239: 8 total, 8 up, 8 in 2024-09-11T08:06:08.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1678854715' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi102-80338-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:08.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:07 smithi146 ceph-mon[33158]: from='client.46928 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi102-80338-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:08.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:07 smithi102 ceph-mon[41371]: pgmap v326: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:06:08.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:07 smithi102 ceph-mon[41371]: from='client.46928 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi102-80338-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:08.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:07 smithi102 ceph-mon[41371]: osdmap e239: 8 total, 8 up, 8 in 2024-09-11T08:06:08.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1678854715' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi102-80338-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:08.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:07 smithi102 ceph-mon[41371]: from='client.46928 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi102-80338-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:08.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:07 smithi102 ceph-mon[29849]: pgmap v326: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:06:08.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:07 smithi102 ceph-mon[29849]: from='client.46928 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi102-80338-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:08.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:07 smithi102 ceph-mon[29849]: osdmap e239: 8 total, 8 up, 8 in 2024-09-11T08:06:08.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1678854715' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi102-80338-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:08.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:07 smithi102 ceph-mon[29849]: from='client.46928 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi102-80338-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:09.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:08 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:09.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:08 smithi102 ceph-mon[29849]: osdmap e240: 8 total, 8 up, 8 in 2024-09-11T08:06:09.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:08 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3777328099' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:09.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:08 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:09.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:08 smithi102 ceph-mon[29849]: from='client.46922 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:09.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:08 smithi102 ceph-mon[29849]: pgmap v329: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:06:09.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:08 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:09.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:08 smithi102 ceph-mon[41371]: osdmap e240: 8 total, 8 up, 8 in 2024-09-11T08:06:09.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:08 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3777328099' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:09.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:08 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:09.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:08 smithi102 ceph-mon[41371]: from='client.46922 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:09.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:08 smithi102 ceph-mon[41371]: pgmap v329: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:06:09.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:08 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:09.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:08 smithi146 ceph-mon[33158]: osdmap e240: 8 total, 8 up, 8 in 2024-09-11T08:06:09.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:08 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3777328099' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:09.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:08 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:09.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:08 smithi146 ceph-mon[33158]: from='client.46922 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:09.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:08 smithi146 ceph-mon[33158]: pgmap v329: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:06:09.933 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:06:09 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:06:10.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:06:09 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:06:09] "GET /metrics HTTP/1.1" 200 228493 "" "Prometheus/2.43.0" 2024-09-11T08:06:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:09 smithi102 ceph-mon[29849]: from='client.46928 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi102-80338-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi102-80338-40"}]': finished 2024-09-11T08:06:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:09 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:09 smithi102 ceph-mon[29849]: from='client.46922 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi102-80356-49"}]': finished 2024-09-11T08:06:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:09 smithi102 ceph-mon[29849]: osdmap e241: 8 total, 8 up, 8 in 2024-09-11T08:06:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:09 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3777328099' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:09 smithi102 ceph-mon[29849]: from='client.46922 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:09 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:09 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:09 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:06:10.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:09 smithi102 ceph-mon[41371]: from='client.46928 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi102-80338-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi102-80338-40"}]': finished 2024-09-11T08:06:10.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:09 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:10.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:09 smithi102 ceph-mon[41371]: from='client.46922 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi102-80356-49"}]': finished 2024-09-11T08:06:10.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:09 smithi102 ceph-mon[41371]: osdmap e241: 8 total, 8 up, 8 in 2024-09-11T08:06:10.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:09 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3777328099' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:10.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:09 smithi102 ceph-mon[41371]: from='client.46922 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:10.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:09 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:10.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:09 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:10.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:09 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:06:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:09 smithi146 ceph-mon[33158]: from='client.46928 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi102-80338-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi102-80338-40"}]': finished 2024-09-11T08:06:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:09 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:09 smithi146 ceph-mon[33158]: from='client.46922 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi102-80356-49"}]': finished 2024-09-11T08:06:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:09 smithi146 ceph-mon[33158]: osdmap e241: 8 total, 8 up, 8 in 2024-09-11T08:06:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:09 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3777328099' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:09 smithi146 ceph-mon[33158]: from='client.46922 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi102-80356-49"}]: dispatch 2024-09-11T08:06:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:09 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:09 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:09 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:06:11.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:10 smithi102 ceph-mon[41371]: from='client.46922 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi102-80356-49"}]': finished 2024-09-11T08:06:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:10 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:06:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:10 smithi102 ceph-mon[41371]: osdmap e242: 8 total, 8 up, 8 in 2024-09-11T08:06:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-33"}]: dispatch 2024-09-11T08:06:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:10 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-33"}]: dispatch 2024-09-11T08:06:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi102-80356-50"}]: dispatch 2024-09-11T08:06:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi102-80356-50"}]: dispatch 2024-09-11T08:06:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi102-80356-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:10 smithi102 ceph-mon[41371]: pgmap v332: 236 pgs: 236 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-11T08:06:11.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:10 smithi102 ceph-mon[29849]: from='client.46922 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi102-80356-49"}]': finished 2024-09-11T08:06:11.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:10 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:06:11.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:10 smithi102 ceph-mon[29849]: osdmap e242: 8 total, 8 up, 8 in 2024-09-11T08:06:11.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-33"}]: dispatch 2024-09-11T08:06:11.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:10 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-33"}]: dispatch 2024-09-11T08:06:11.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi102-80356-50"}]: dispatch 2024-09-11T08:06:11.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi102-80356-50"}]: dispatch 2024-09-11T08:06:11.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi102-80356-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:11.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:10 smithi102 ceph-mon[29849]: pgmap v332: 236 pgs: 236 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-11T08:06:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:10 smithi146 ceph-mon[33158]: from='client.46922 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi102-80356-49"}]': finished 2024-09-11T08:06:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:10 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:06:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:10 smithi146 ceph-mon[33158]: osdmap e242: 8 total, 8 up, 8 in 2024-09-11T08:06:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-33"}]: dispatch 2024-09-11T08:06:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:10 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-33"}]: dispatch 2024-09-11T08:06:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi102-80356-50"}]: dispatch 2024-09-11T08:06:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi102-80356-50"}]: dispatch 2024-09-11T08:06:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi102-80356-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:10 smithi146 ceph-mon[33158]: pgmap v332: 236 pgs: 236 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-11T08:06:12.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:12 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-33"}]': finished 2024-09-11T08:06:12.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi102-80356-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:12.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:12 smithi102 ceph-mon[29849]: osdmap e243: 8 total, 8 up, 8 in 2024-09-11T08:06:12.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-33", "mode": "writeback"}]: dispatch 2024-09-11T08:06:12.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi102-80356-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi102-80356-50"}]: dispatch 2024-09-11T08:06:12.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:12 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-33", "mode": "writeback"}]: dispatch 2024-09-11T08:06:12.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1678854715' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:12.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:12 smithi102 ceph-mon[29849]: from='client.46928 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:12.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:12 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:12.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:12 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-33"}]': finished 2024-09-11T08:06:12.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi102-80356-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:12.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:12 smithi102 ceph-mon[41371]: osdmap e243: 8 total, 8 up, 8 in 2024-09-11T08:06:12.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-33", "mode": "writeback"}]: dispatch 2024-09-11T08:06:12.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi102-80356-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi102-80356-50"}]: dispatch 2024-09-11T08:06:12.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:12 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-33", "mode": "writeback"}]: dispatch 2024-09-11T08:06:12.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1678854715' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:12.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:12 smithi102 ceph-mon[41371]: from='client.46928 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:12.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:12 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:12.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:12 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-33"}]': finished 2024-09-11T08:06:12.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi102-80356-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:12.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:12 smithi146 ceph-mon[33158]: osdmap e243: 8 total, 8 up, 8 in 2024-09-11T08:06:12.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-33", "mode": "writeback"}]: dispatch 2024-09-11T08:06:12.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi102-80356-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi102-80356-50"}]: dispatch 2024-09-11T08:06:12.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:12 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-33", "mode": "writeback"}]: dispatch 2024-09-11T08:06:12.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1678854715' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:12.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:12 smithi146 ceph-mon[33158]: from='client.46928 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:12.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:12 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:13.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:06:13.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-33", "mode": "writeback"}]': finished 2024-09-11T08:06:13.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[41371]: from='client.46928 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi102-80338-40"}]': finished 2024-09-11T08:06:13.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[41371]: osdmap e244: 8 total, 8 up, 8 in 2024-09-11T08:06:13.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1678854715' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:13.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[41371]: from='client.46928 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:13.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:13.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:13.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[41371]: pgmap v335: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:13.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi102-80356-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi102-80356-50"}]': finished 2024-09-11T08:06:13.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[41371]: from='client.46928 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi102-80338-40"}]': finished 2024-09-11T08:06:13.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:06:13.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[41371]: osdmap e245: 8 total, 8 up, 8 in 2024-09-11T08:06:13.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-33"}]: dispatch 2024-09-11T08:06:13.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-33"}]: dispatch 2024-09-11T08:06:13.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi102-80338-41"}]: dispatch 2024-09-11T08:06:13.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi102-80338-41"}]: dispatch 2024-09-11T08:06:13.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi102-80338-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:13.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:06:13.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-33", "mode": "writeback"}]': finished 2024-09-11T08:06:13.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[29849]: from='client.46928 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi102-80338-40"}]': finished 2024-09-11T08:06:13.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[29849]: osdmap e244: 8 total, 8 up, 8 in 2024-09-11T08:06:13.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1678854715' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:13.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[29849]: from='client.46928 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:13.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:13.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:13.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[29849]: pgmap v335: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:13.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi102-80356-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi102-80356-50"}]': finished 2024-09-11T08:06:13.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[29849]: from='client.46928 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi102-80338-40"}]': finished 2024-09-11T08:06:13.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:06:13.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[29849]: osdmap e245: 8 total, 8 up, 8 in 2024-09-11T08:06:13.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-33"}]: dispatch 2024-09-11T08:06:13.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-33"}]: dispatch 2024-09-11T08:06:13.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi102-80338-41"}]: dispatch 2024-09-11T08:06:13.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi102-80338-41"}]: dispatch 2024-09-11T08:06:13.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi102-80338-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:13.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:13 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:06:13.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:13 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-33", "mode": "writeback"}]': finished 2024-09-11T08:06:13.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:13 smithi146 ceph-mon[33158]: from='client.46928 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi102-80338-40"}]': finished 2024-09-11T08:06:13.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:13 smithi146 ceph-mon[33158]: osdmap e244: 8 total, 8 up, 8 in 2024-09-11T08:06:13.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1678854715' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:13.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:13 smithi146 ceph-mon[33158]: from='client.46928 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi102-80338-40"}]: dispatch 2024-09-11T08:06:13.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:13.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:13 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:13 smithi146 ceph-mon[33158]: pgmap v335: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi102-80356-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi102-80356-50"}]': finished 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:13 smithi146 ceph-mon[33158]: from='client.46928 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi102-80338-40"}]': finished 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:13 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:13 smithi146 ceph-mon[33158]: osdmap e245: 8 total, 8 up, 8 in 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-33"}]: dispatch 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:13 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-33"}]: dispatch 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi102-80338-41"}]: dispatch 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi102-80338-41"}]: dispatch 2024-09-11T08:06:13.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi102-80338-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:14.588 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:14 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:06:14.588 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:14 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-33"}]': finished 2024-09-11T08:06:14.588 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi102-80338-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:14.588 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:14 smithi146 ceph-mon[33158]: osdmap e246: 8 total, 8 up, 8 in 2024-09-11T08:06:14.588 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi102-80338-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi102-80338-41"}]: dispatch 2024-09-11T08:06:14.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:14 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:06:14.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:14 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-33"}]': finished 2024-09-11T08:06:14.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi102-80338-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:14.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:14 smithi102 ceph-mon[41371]: osdmap e246: 8 total, 8 up, 8 in 2024-09-11T08:06:14.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi102-80338-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi102-80338-41"}]: dispatch 2024-09-11T08:06:14.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:14 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:06:14.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:14 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-33"}]': finished 2024-09-11T08:06:14.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi102-80338-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:14.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:14 smithi102 ceph-mon[29849]: osdmap e246: 8 total, 8 up, 8 in 2024-09-11T08:06:14.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi102-80338-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi102-80338-41"}]: dispatch 2024-09-11T08:06:14.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:06:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:06:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:06:15.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:06:15.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:15 smithi102 ceph-mon[41371]: pgmap v338: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:15.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:06:15.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:15 smithi102 ceph-mon[29849]: pgmap v338: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:15.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:06:15.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:15 smithi146 ceph-mon[33158]: pgmap v338: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:16 smithi102 ceph-mon[29849]: osdmap e247: 8 total, 8 up, 8 in 2024-09-11T08:06:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi102-80356-50"}]: dispatch 2024-09-11T08:06:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi102-80338-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi102-80338-41"}]': finished 2024-09-11T08:06:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi102-80356-50"}]': finished 2024-09-11T08:06:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:16 smithi102 ceph-mon[29849]: osdmap e248: 8 total, 8 up, 8 in 2024-09-11T08:06:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi102-80356-50"}]: dispatch 2024-09-11T08:06:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:16 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:16.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:16 smithi102 ceph-mon[41371]: osdmap e247: 8 total, 8 up, 8 in 2024-09-11T08:06:16.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi102-80356-50"}]: dispatch 2024-09-11T08:06:16.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi102-80338-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi102-80338-41"}]': finished 2024-09-11T08:06:16.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi102-80356-50"}]': finished 2024-09-11T08:06:16.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:16 smithi102 ceph-mon[41371]: osdmap e248: 8 total, 8 up, 8 in 2024-09-11T08:06:16.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi102-80356-50"}]: dispatch 2024-09-11T08:06:16.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:16.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:16 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:16.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:16 smithi146 ceph-mon[33158]: osdmap e247: 8 total, 8 up, 8 in 2024-09-11T08:06:16.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi102-80356-50"}]: dispatch 2024-09-11T08:06:16.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi102-80338-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi102-80338-41"}]': finished 2024-09-11T08:06:16.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi102-80356-50"}]': finished 2024-09-11T08:06:16.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:16 smithi146 ceph-mon[33158]: osdmap e248: 8 total, 8 up, 8 in 2024-09-11T08:06:16.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi102-80356-50"}]: dispatch 2024-09-11T08:06:16.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:16.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:16 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:17.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:17 smithi102 ceph-mon[29849]: pgmap v341: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:06:17.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:17 smithi102 ceph-mon[29849]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:17.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi102-80356-50"}]': finished 2024-09-11T08:06:17.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:17 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:17.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:17 smithi102 ceph-mon[29849]: osdmap e249: 8 total, 8 up, 8 in 2024-09-11T08:06:17.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:17.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:17 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:17.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:17 smithi102 ceph-mon[41371]: pgmap v341: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:06:17.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:17 smithi102 ceph-mon[41371]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:17.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi102-80356-50"}]': finished 2024-09-11T08:06:17.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:17 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:17.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:17 smithi102 ceph-mon[41371]: osdmap e249: 8 total, 8 up, 8 in 2024-09-11T08:06:17.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:17.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:17 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:17.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:17 smithi146 ceph-mon[33158]: pgmap v341: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:06:17.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:17 smithi146 ceph-mon[33158]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:17.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2994439931' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi102-80356-50"}]': finished 2024-09-11T08:06:17.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:17 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:17.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:17 smithi146 ceph-mon[33158]: osdmap e249: 8 total, 8 up, 8 in 2024-09-11T08:06:17.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:17.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:17 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:19.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:18 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:06:19.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:18 smithi102 ceph-mon[29849]: osdmap e250: 8 total, 8 up, 8 in 2024-09-11T08:06:19.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-35"}]: dispatch 2024-09-11T08:06:19.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:18 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-35"}]: dispatch 2024-09-11T08:06:19.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3725184910' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi102-80356-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:19.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi102-80338-41"}]: dispatch 2024-09-11T08:06:19.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:18 smithi102 ceph-mon[29849]: from='client.46946 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi102-80356-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:19.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:18 smithi102 ceph-mon[29849]: pgmap v344: 260 pgs: 64 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:06:19.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:18 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:06:19.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:18 smithi102 ceph-mon[41371]: osdmap e250: 8 total, 8 up, 8 in 2024-09-11T08:06:19.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-35"}]: dispatch 2024-09-11T08:06:19.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:18 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-35"}]: dispatch 2024-09-11T08:06:19.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3725184910' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi102-80356-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:19.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi102-80338-41"}]: dispatch 2024-09-11T08:06:19.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:18 smithi102 ceph-mon[41371]: from='client.46946 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi102-80356-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:19.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:18 smithi102 ceph-mon[41371]: pgmap v344: 260 pgs: 64 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:06:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:18 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:06:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:18 smithi146 ceph-mon[33158]: osdmap e250: 8 total, 8 up, 8 in 2024-09-11T08:06:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-35"}]: dispatch 2024-09-11T08:06:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:18 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-35"}]: dispatch 2024-09-11T08:06:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3725184910' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi102-80356-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi102-80338-41"}]: dispatch 2024-09-11T08:06:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:18 smithi146 ceph-mon[33158]: from='client.46946 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi102-80356-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:18 smithi146 ceph-mon[33158]: pgmap v344: 260 pgs: 64 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:06:19.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:06:19 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:06:19.992 INFO:tasks.workunit.client.0.smithi102.stdout:te 2024-09-11T08:06:19.992 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (6850 ms) 2024-09-11T08:06:19.992 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-09-11T08:06:19.992 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (7042 ms) 2024-09-11T08:06:19.993 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-09-11T08:06:19.993 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (6975 ms) 2024-09-11T08:06:19.993 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-09-11T08:06:19.993 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7274 ms) 2024-09-11T08:06:19.993 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-09-11T08:06:19.993 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (6810 ms) 2024-09-11T08:06:19.993 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-09-11T08:06:19.993 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7030 ms) 2024-09-11T08:06:19.993 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-09-11T08:06:19.993 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7029 ms) 2024-09-11T08:06:19.993 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-09-11T08:06:19.993 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (7022 ms) 2024-09-11T08:06:19.994 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-09-11T08:06:19.994 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (6831 ms) 2024-09-11T08:06:19.994 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-09-11T08:06:19.994 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (7036 ms) 2024-09-11T08:06:19.994 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-09-11T08:06:19.994 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (6960 ms) 2024-09-11T08:06:19.994 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-09-11T08:06:19.994 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (7026 ms) 2024-09-11T08:06:19.994 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-09-11T08:06:19.994 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7003 ms) 2024-09-11T08:06:19.994 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-09-11T08:06:19.994 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7039 ms) 2024-09-11T08:06:19.994 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-09-11T08:06:19.994 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (7023 ms) 2024-09-11T08:06:19.995 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (111995 ms total) 2024-09-11T08:06:19.995 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: 2024-09-11T08:06:19.995 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [----------] Global test environment tear-down 2024-09-11T08:06:19.995 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [==========] 41 tests from 2 test suites ran. (191857 ms total) 2024-09-11T08:06:19.995 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio: [ PASSED ] 41 tests. 2024-09-11T08:06:20.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:06:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:06:19] "GET /metrics HTTP/1.1" 200 230918 "" "Prometheus/2.43.0" 2024-09-11T08:06:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:19 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-35"}]': finished 2024-09-11T08:06:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi102-80338-41"}]': finished 2024-09-11T08:06:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:19 smithi102 ceph-mon[29849]: from='client.46946 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi102-80356-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:19 smithi102 ceph-mon[29849]: osdmap e251: 8 total, 8 up, 8 in 2024-09-11T08:06:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-35", "mode": "writeback"}]: dispatch 2024-09-11T08:06:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi102-80338-41"}]: dispatch 2024-09-11T08:06:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:19 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-35", "mode": "writeback"}]: dispatch 2024-09-11T08:06:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:19 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:06:20.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:19 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-35"}]': finished 2024-09-11T08:06:20.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi102-80338-41"}]': finished 2024-09-11T08:06:20.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:19 smithi102 ceph-mon[41371]: from='client.46946 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi102-80356-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:20.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:19 smithi102 ceph-mon[41371]: osdmap e251: 8 total, 8 up, 8 in 2024-09-11T08:06:20.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-35", "mode": "writeback"}]: dispatch 2024-09-11T08:06:20.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi102-80338-41"}]: dispatch 2024-09-11T08:06:20.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:19 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-35", "mode": "writeback"}]: dispatch 2024-09-11T08:06:20.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:19 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:06:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:19 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-35"}]': finished 2024-09-11T08:06:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi102-80338-41"}]': finished 2024-09-11T08:06:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:19 smithi146 ceph-mon[33158]: from='client.46946 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi102-80356-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:19 smithi146 ceph-mon[33158]: osdmap e251: 8 total, 8 up, 8 in 2024-09-11T08:06:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-35", "mode": "writeback"}]: dispatch 2024-09-11T08:06:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi102-80338-41"}]: dispatch 2024-09-11T08:06:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:19 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-35", "mode": "writeback"}]: dispatch 2024-09-11T08:06:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:19 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:06:21.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:20 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:06:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi102-80338-41"}]': finished 2024-09-11T08:06:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:20 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-35", "mode": "writeback"}]': finished 2024-09-11T08:06:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:20 smithi102 ceph-mon[29849]: osdmap e252: 8 total, 8 up, 8 in 2024-09-11T08:06:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi102-80356-52"}]: dispatch 2024-09-11T08:06:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi102-80356-52"}]: dispatch 2024-09-11T08:06:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi102-80356-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:20 smithi102 ceph-mon[29849]: pgmap v347: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:06:21.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:20 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:06:21.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi102-80338-41"}]': finished 2024-09-11T08:06:21.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:20 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-35", "mode": "writeback"}]': finished 2024-09-11T08:06:21.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:20 smithi102 ceph-mon[41371]: osdmap e252: 8 total, 8 up, 8 in 2024-09-11T08:06:21.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi102-80356-52"}]: dispatch 2024-09-11T08:06:21.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi102-80356-52"}]: dispatch 2024-09-11T08:06:21.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi102-80356-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:21.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:20 smithi102 ceph-mon[41371]: pgmap v347: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:06:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:20 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:06:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/161255029' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi102-80338-41"}]': finished 2024-09-11T08:06:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:20 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-35", "mode": "writeback"}]': finished 2024-09-11T08:06:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:20 smithi146 ceph-mon[33158]: osdmap e252: 8 total, 8 up, 8 in 2024-09-11T08:06:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi102-80356-52"}]: dispatch 2024-09-11T08:06:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi102-80356-52"}]: dispatch 2024-09-11T08:06:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi102-80356-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:20 smithi146 ceph-mon[33158]: pgmap v347: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:06:22.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi102-80356-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:22.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:22 smithi102 ceph-mon[29849]: osdmap e253: 8 total, 8 up, 8 in 2024-09-11T08:06:22.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi102-80356-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi102-80356-52"}]: dispatch 2024-09-11T08:06:22.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi102-80356-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:22.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:22 smithi102 ceph-mon[41371]: osdmap e253: 8 total, 8 up, 8 in 2024-09-11T08:06:22.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi102-80356-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi102-80356-52"}]: dispatch 2024-09-11T08:06:22.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi102-80356-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:22.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:22 smithi146 ceph-mon[33158]: osdmap e253: 8 total, 8 up, 8 in 2024-09-11T08:06:22.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi102-80356-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi102-80356-52"}]: dispatch 2024-09-11T08:06:23.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:23 smithi102 ceph-mon[29849]: osdmap e254: 8 total, 8 up, 8 in 2024-09-11T08:06:23.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:23.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:23 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:23.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:23 smithi102 ceph-mon[29849]: pgmap v350: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:06:23.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi102-80356-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi102-80356-52"}]': finished 2024-09-11T08:06:23.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:23 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:06:23.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:23 smithi102 ceph-mon[29849]: osdmap e255: 8 total, 8 up, 8 in 2024-09-11T08:06:23.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-35"}]: dispatch 2024-09-11T08:06:23.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:23 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-35"}]: dispatch 2024-09-11T08:06:23.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:23 smithi102 ceph-mon[41371]: osdmap e254: 8 total, 8 up, 8 in 2024-09-11T08:06:23.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:23.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:23 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:23.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:23 smithi102 ceph-mon[41371]: pgmap v350: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:06:23.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi102-80356-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi102-80356-52"}]': finished 2024-09-11T08:06:23.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:23 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:06:23.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:23 smithi102 ceph-mon[41371]: osdmap e255: 8 total, 8 up, 8 in 2024-09-11T08:06:23.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-35"}]: dispatch 2024-09-11T08:06:23.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:23 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-35"}]: dispatch 2024-09-11T08:06:23.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:23 smithi146 ceph-mon[33158]: osdmap e254: 8 total, 8 up, 8 in 2024-09-11T08:06:23.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:23.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:23 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:23.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:23 smithi146 ceph-mon[33158]: pgmap v350: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:06:23.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi102-80356-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi102-80356-52"}]': finished 2024-09-11T08:06:23.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:23 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:06:23.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:23 smithi146 ceph-mon[33158]: osdmap e255: 8 total, 8 up, 8 in 2024-09-11T08:06:23.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-35"}]: dispatch 2024-09-11T08:06:23.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:23 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-35"}]: dispatch 2024-09-11T08:06:24.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:06:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:06:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:06:25.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:25 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-35"}]': finished 2024-09-11T08:06:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:25 smithi102 ceph-mon[29849]: osdmap e256: 8 total, 8 up, 8 in 2024-09-11T08:06:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:25 smithi102 ceph-mon[29849]: pgmap v353: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:25.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:25 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-35"}]': finished 2024-09-11T08:06:25.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:25 smithi102 ceph-mon[41371]: osdmap e256: 8 total, 8 up, 8 in 2024-09-11T08:06:25.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:25 smithi102 ceph-mon[41371]: pgmap v353: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:25 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-35"}]': finished 2024-09-11T08:06:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:25 smithi146 ceph-mon[33158]: osdmap e256: 8 total, 8 up, 8 in 2024-09-11T08:06:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:25 smithi146 ceph-mon[33158]: pgmap v353: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:26.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:26 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:26.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:26 smithi102 ceph-mon[29849]: osdmap e257: 8 total, 8 up, 8 in 2024-09-11T08:06:26.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi102-80356-52"}]: dispatch 2024-09-11T08:06:26.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:26.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:26 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:26.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:26 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:26.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:26 smithi102 ceph-mon[41371]: osdmap e257: 8 total, 8 up, 8 in 2024-09-11T08:06:26.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi102-80356-52"}]: dispatch 2024-09-11T08:06:26.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:26.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:26 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:26.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:26 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:26.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:26 smithi146 ceph-mon[33158]: osdmap e257: 8 total, 8 up, 8 in 2024-09-11T08:06:26.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi102-80356-52"}]: dispatch 2024-09-11T08:06:26.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:26.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:26 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:27.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi102-80356-52"}]': finished 2024-09-11T08:06:27.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:27 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:06:27.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:27 smithi102 ceph-mon[41371]: osdmap e258: 8 total, 8 up, 8 in 2024-09-11T08:06:27.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-35"}]: dispatch 2024-09-11T08:06:27.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:27 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-35"}]: dispatch 2024-09-11T08:06:27.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi102-80356-52"}]: dispatch 2024-09-11T08:06:27.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:27 smithi102 ceph-mon[41371]: pgmap v356: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-11T08:06:27.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:27 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:06:27.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:27 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-35"}]': finished 2024-09-11T08:06:27.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi102-80356-52"}]': finished 2024-09-11T08:06:27.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:27 smithi102 ceph-mon[41371]: osdmap e259: 8 total, 8 up, 8 in 2024-09-11T08:06:27.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi102-80356-52"}]': finished 2024-09-11T08:06:27.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:27 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:06:27.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:27 smithi102 ceph-mon[29849]: osdmap e258: 8 total, 8 up, 8 in 2024-09-11T08:06:27.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-35"}]: dispatch 2024-09-11T08:06:27.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:27 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-35"}]: dispatch 2024-09-11T08:06:27.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi102-80356-52"}]: dispatch 2024-09-11T08:06:27.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:27 smithi102 ceph-mon[29849]: pgmap v356: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-11T08:06:27.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:27 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:06:27.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:27 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-35"}]': finished 2024-09-11T08:06:27.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi102-80356-52"}]': finished 2024-09-11T08:06:27.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:27 smithi102 ceph-mon[29849]: osdmap e259: 8 total, 8 up, 8 in 2024-09-11T08:06:27.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi102-80356-52"}]': finished 2024-09-11T08:06:27.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:27 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:06:27.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:27 smithi146 ceph-mon[33158]: osdmap e258: 8 total, 8 up, 8 in 2024-09-11T08:06:27.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-35"}]: dispatch 2024-09-11T08:06:27.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:27 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-35"}]: dispatch 2024-09-11T08:06:27.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi102-80356-52"}]: dispatch 2024-09-11T08:06:27.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:27 smithi146 ceph-mon[33158]: pgmap v356: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-11T08:06:27.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:27 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:06:27.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:27 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-35"}]': finished 2024-09-11T08:06:27.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2073241168' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi102-80356-52"}]': finished 2024-09-11T08:06:27.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:27 smithi146 ceph-mon[33158]: osdmap e259: 8 total, 8 up, 8 in 2024-09-11T08:06:28.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1715041895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:28.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:28 smithi102 ceph-mon[29849]: from='client.48061 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:28.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1715041895' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:28.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:28 smithi102 ceph-mon[29849]: from='client.48061 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:28.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1715041895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:28.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:28 smithi102 ceph-mon[29849]: from='client.48061 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:28.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1715041895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:28.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:28 smithi102 ceph-mon[41371]: from='client.48061 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:28.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1715041895' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:28.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:28 smithi102 ceph-mon[41371]: from='client.48061 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:28.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1715041895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:28.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:28 smithi102 ceph-mon[41371]: from='client.48061 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:28.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1715041895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:28.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:28 smithi146 ceph-mon[33158]: from='client.48061 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:28.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1715041895' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:28.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:28 smithi146 ceph-mon[33158]: from='client.48061 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:28.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1715041895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:28.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:28 smithi146 ceph-mon[33158]: from='client.48061 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:29.610 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:29 smithi146 ceph-mon[33158]: from='client.48061 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:29.610 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:29 smithi146 ceph-mon[33158]: osdmap e260: 8 total, 8 up, 8 in 2024-09-11T08:06:29.610 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1715041895' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi102-80356-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:29.610 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:29 smithi146 ceph-mon[33158]: from='client.48061 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi102-80356-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:29.610 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:29 smithi146 ceph-mon[33158]: pgmap v359: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:06:29.610 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:29 smithi146 ceph-mon[33158]: osdmap e261: 8 total, 8 up, 8 in 2024-09-11T08:06:29.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:29 smithi102 ceph-mon[41371]: from='client.48061 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:29.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:29 smithi102 ceph-mon[41371]: osdmap e260: 8 total, 8 up, 8 in 2024-09-11T08:06:29.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1715041895' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi102-80356-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:29.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:29 smithi102 ceph-mon[41371]: from='client.48061 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi102-80356-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:29.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:29 smithi102 ceph-mon[41371]: pgmap v359: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:06:29.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:29 smithi102 ceph-mon[41371]: osdmap e261: 8 total, 8 up, 8 in 2024-09-11T08:06:29.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:29 smithi102 ceph-mon[29849]: from='client.48061 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:29.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:29 smithi102 ceph-mon[29849]: osdmap e260: 8 total, 8 up, 8 in 2024-09-11T08:06:29.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1715041895' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi102-80356-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:29.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:29 smithi102 ceph-mon[29849]: from='client.48061 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi102-80356-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:29.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:29 smithi102 ceph-mon[29849]: pgmap v359: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:06:29.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:29 smithi102 ceph-mon[29849]: osdmap e261: 8 total, 8 up, 8 in 2024-09-11T08:06:29.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:06:29 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:06:30.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:06:29 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:06:29] "GET /metrics HTTP/1.1" 200 228477 "" "Prometheus/2.43.0" 2024-09-11T08:06:30.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:30 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:30.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:30 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:30.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:30 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:06:30.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:30 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:06:30.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:30 smithi102 ceph-mon[29849]: from='client.48061 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi102-80356-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi102-80356-53"}]': finished 2024-09-11T08:06:30.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:30 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:30.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:30 smithi102 ceph-mon[29849]: osdmap e262: 8 total, 8 up, 8 in 2024-09-11T08:06:30.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:30 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:30.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:30 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:30.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:30 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:30.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:30 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:30.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:30 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:06:30.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:30 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:06:30.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:30 smithi102 ceph-mon[41371]: from='client.48061 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi102-80356-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi102-80356-53"}]': finished 2024-09-11T08:06:30.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:30 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:30.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:30 smithi102 ceph-mon[41371]: osdmap e262: 8 total, 8 up, 8 in 2024-09-11T08:06:30.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:30 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:30.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:30 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:30.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:30 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:30.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:30 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:30.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:30 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:06:30.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:30 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:06:30.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:30 smithi146 ceph-mon[33158]: from='client.48061 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi102-80356-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi102-80356-53"}]': finished 2024-09-11T08:06:30.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:30 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:30.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:30 smithi146 ceph-mon[33158]: osdmap e262: 8 total, 8 up, 8 in 2024-09-11T08:06:30.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:30 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:30.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:30 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:31.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:31 smithi102 ceph-mon[41371]: pgmap v362: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-11T08:06:31.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:31 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:06:31.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:31 smithi102 ceph-mon[41371]: osdmap e263: 8 total, 8 up, 8 in 2024-09-11T08:06:31.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:31 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-37"}]: dispatch 2024-09-11T08:06:31.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:31 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-37"}]: dispatch 2024-09-11T08:06:31.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:31 smithi102 ceph-mon[29849]: pgmap v362: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-11T08:06:31.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:31 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:06:31.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:31 smithi102 ceph-mon[29849]: osdmap e263: 8 total, 8 up, 8 in 2024-09-11T08:06:31.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:31 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-37"}]: dispatch 2024-09-11T08:06:31.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:31 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-37"}]: dispatch 2024-09-11T08:06:31.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:31 smithi146 ceph-mon[33158]: pgmap v362: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-11T08:06:31.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:31 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:06:31.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:31 smithi146 ceph-mon[33158]: osdmap e263: 8 total, 8 up, 8 in 2024-09-11T08:06:31.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:31 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-37"}]: dispatch 2024-09-11T08:06:31.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:31 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-37"}]: dispatch 2024-09-11T08:06:32.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:32 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:32.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:32 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-37"}]': finished 2024-09-11T08:06:32.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:32 smithi102 ceph-mon[41371]: osdmap e264: 8 total, 8 up, 8 in 2024-09-11T08:06:32.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-37", "mode": "writeback"}]: dispatch 2024-09-11T08:06:32.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:32 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-37", "mode": "writeback"}]: dispatch 2024-09-11T08:06:32.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1715041895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:32.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:32 smithi102 ceph-mon[41371]: from='client.48061 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:32.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:32 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:32.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:32 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-37"}]': finished 2024-09-11T08:06:32.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:32 smithi102 ceph-mon[29849]: osdmap e264: 8 total, 8 up, 8 in 2024-09-11T08:06:32.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-37", "mode": "writeback"}]: dispatch 2024-09-11T08:06:32.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:32 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-37", "mode": "writeback"}]: dispatch 2024-09-11T08:06:32.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1715041895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:32.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:32 smithi102 ceph-mon[29849]: from='client.48061 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:32.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:32 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:32.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:32 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-37"}]': finished 2024-09-11T08:06:32.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:32 smithi146 ceph-mon[33158]: osdmap e264: 8 total, 8 up, 8 in 2024-09-11T08:06:32.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-37", "mode": "writeback"}]: dispatch 2024-09-11T08:06:32.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:32 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-37", "mode": "writeback"}]: dispatch 2024-09-11T08:06:32.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1715041895' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:32.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:32 smithi146 ceph-mon[33158]: from='client.48061 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:33.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:33 smithi102 ceph-mon[41371]: pgmap v365: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:33.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:33 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:06:33.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:33 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-37", "mode": "writeback"}]': finished 2024-09-11T08:06:33.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:33 smithi102 ceph-mon[41371]: from='client.48061 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53"}]': finished 2024-09-11T08:06:33.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:33 smithi102 ceph-mon[41371]: osdmap e265: 8 total, 8 up, 8 in 2024-09-11T08:06:33.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1715041895' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:33.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:33 smithi102 ceph-mon[41371]: from='client.48061 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:33.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:33.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:33 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:33.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:33 smithi102 ceph-mon[29849]: pgmap v365: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:33.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:33 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:06:33.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:33 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-37", "mode": "writeback"}]': finished 2024-09-11T08:06:33.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:33 smithi102 ceph-mon[29849]: from='client.48061 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53"}]': finished 2024-09-11T08:06:33.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:33 smithi102 ceph-mon[29849]: osdmap e265: 8 total, 8 up, 8 in 2024-09-11T08:06:33.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1715041895' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:33.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:33 smithi102 ceph-mon[29849]: from='client.48061 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:33.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:33.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:33 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:33.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:33 smithi146 ceph-mon[33158]: pgmap v365: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:33.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:33 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:06:33.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:33 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-37", "mode": "writeback"}]': finished 2024-09-11T08:06:33.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:33 smithi146 ceph-mon[33158]: from='client.48061 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi102-80356-53"}]': finished 2024-09-11T08:06:33.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:33 smithi146 ceph-mon[33158]: osdmap e265: 8 total, 8 up, 8 in 2024-09-11T08:06:33.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1715041895' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:33.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:33 smithi146 ceph-mon[33158]: from='client.48061 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi102-80356-53"}]: dispatch 2024-09-11T08:06:33.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:33.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:33 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:34.908 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:06:34 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:06:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:06:35.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:34 smithi146 ceph-mon[33158]: from='client.48061 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi102-80356-53"}]': finished 2024-09-11T08:06:35.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:34 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:06:35.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:34 smithi146 ceph-mon[33158]: osdmap e266: 8 total, 8 up, 8 in 2024-09-11T08:06:35.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:34 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-37"}]: dispatch 2024-09-11T08:06:35.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:34 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-37"}]: dispatch 2024-09-11T08:06:35.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:34 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi102-80356-54"}]: dispatch 2024-09-11T08:06:35.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:34 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi102-80356-54"}]: dispatch 2024-09-11T08:06:35.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:34 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi102-80356-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:35.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:34 smithi102 ceph-mon[41371]: from='client.48061 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi102-80356-53"}]': finished 2024-09-11T08:06:35.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:34 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:06:35.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:34 smithi102 ceph-mon[41371]: osdmap e266: 8 total, 8 up, 8 in 2024-09-11T08:06:35.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:34 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-37"}]: dispatch 2024-09-11T08:06:35.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:34 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-37"}]: dispatch 2024-09-11T08:06:35.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:34 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi102-80356-54"}]: dispatch 2024-09-11T08:06:35.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:34 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi102-80356-54"}]: dispatch 2024-09-11T08:06:35.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:34 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi102-80356-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:34 smithi102 ceph-mon[29849]: from='client.48061 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi102-80356-53"}]': finished 2024-09-11T08:06:35.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:34 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:06:35.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:34 smithi102 ceph-mon[29849]: osdmap e266: 8 total, 8 up, 8 in 2024-09-11T08:06:35.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:34 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-37"}]: dispatch 2024-09-11T08:06:35.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:34 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-37"}]: dispatch 2024-09-11T08:06:35.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:34 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi102-80356-54"}]: dispatch 2024-09-11T08:06:35.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:34 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi102-80356-54"}]: dispatch 2024-09-11T08:06:35.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:34 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi102-80356-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:36 smithi146 ceph-mon[33158]: pgmap v368: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:36 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:06:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:36 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-37"}]': finished 2024-09-11T08:06:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:36 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi102-80356-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:36 smithi146 ceph-mon[33158]: osdmap e267: 8 total, 8 up, 8 in 2024-09-11T08:06:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:36 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi102-80356-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi102-80356-54"}]: dispatch 2024-09-11T08:06:36.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:36 smithi102 ceph-mon[41371]: pgmap v368: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:36.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:36 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:06:36.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:36 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-37"}]': finished 2024-09-11T08:06:36.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:36 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi102-80356-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:36.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:36 smithi102 ceph-mon[41371]: osdmap e267: 8 total, 8 up, 8 in 2024-09-11T08:06:36.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:36 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi102-80356-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi102-80356-54"}]: dispatch 2024-09-11T08:06:36.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:36 smithi102 ceph-mon[29849]: pgmap v368: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:36.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:36 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:06:36.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:36 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-37"}]': finished 2024-09-11T08:06:36.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:36 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi102-80356-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:36.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:36 smithi102 ceph-mon[29849]: osdmap e267: 8 total, 8 up, 8 in 2024-09-11T08:06:36.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:36 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi102-80356-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi102-80356-54"}]: dispatch 2024-09-11T08:06:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:37 smithi146 ceph-mon[33158]: osdmap e268: 8 total, 8 up, 8 in 2024-09-11T08:06:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:37 smithi146 ceph-mon[33158]: pgmap v371: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:06:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:37 smithi146 ceph-mon[33158]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi102-80356-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi102-80356-54"}]': finished 2024-09-11T08:06:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:37 smithi146 ceph-mon[33158]: osdmap e269: 8 total, 8 up, 8 in 2024-09-11T08:06:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:37 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:37.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:37 smithi102 ceph-mon[41371]: osdmap e268: 8 total, 8 up, 8 in 2024-09-11T08:06:37.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:37 smithi102 ceph-mon[41371]: pgmap v371: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:06:37.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:37 smithi102 ceph-mon[41371]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:37.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi102-80356-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi102-80356-54"}]': finished 2024-09-11T08:06:37.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:37 smithi102 ceph-mon[41371]: osdmap e269: 8 total, 8 up, 8 in 2024-09-11T08:06:37.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:37.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:37 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:37.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:37 smithi102 ceph-mon[29849]: osdmap e268: 8 total, 8 up, 8 in 2024-09-11T08:06:37.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:37 smithi102 ceph-mon[29849]: pgmap v371: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:06:37.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:37 smithi102 ceph-mon[29849]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:37.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi102-80356-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi102-80356-54"}]': finished 2024-09-11T08:06:37.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:37 smithi102 ceph-mon[29849]: osdmap e269: 8 total, 8 up, 8 in 2024-09-11T08:06:37.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:37.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:37 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:38 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:38 smithi146 ceph-mon[33158]: osdmap e270: 8 total, 8 up, 8 in 2024-09-11T08:06:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:38 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:39.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:38 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:39.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:38 smithi102 ceph-mon[41371]: osdmap e270: 8 total, 8 up, 8 in 2024-09-11T08:06:39.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:39.399 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:38 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:39.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:38 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:39.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:38 smithi102 ceph-mon[29849]: osdmap e270: 8 total, 8 up, 8 in 2024-09-11T08:06:39.399 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:39.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:38 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:39.931 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:06:39 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:06:40.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:06:39 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:06:39] "GET /metrics HTTP/1.1" 200 228477 "" "Prometheus/2.43.0" 2024-09-11T08:06:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:39 smithi102 ceph-mon[41371]: pgmap v374: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:06:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:39 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:06:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:39 smithi102 ceph-mon[41371]: osdmap e271: 8 total, 8 up, 8 in 2024-09-11T08:06:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:39 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-39"}]: dispatch 2024-09-11T08:06:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:39 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi102-80356-54"}]: dispatch 2024-09-11T08:06:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:39 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-39"}]: dispatch 2024-09-11T08:06:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:39 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:40.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:39 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:06:40.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:39 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:06:40.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:39 smithi102 ceph-mon[29849]: pgmap v374: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:06:40.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:39 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:06:40.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:39 smithi102 ceph-mon[29849]: osdmap e271: 8 total, 8 up, 8 in 2024-09-11T08:06:40.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:39 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-39"}]: dispatch 2024-09-11T08:06:40.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:39 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi102-80356-54"}]: dispatch 2024-09-11T08:06:40.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:39 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-39"}]: dispatch 2024-09-11T08:06:40.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:39 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:40.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:39 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:06:40.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:39 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:06:40.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:39 smithi146 ceph-mon[33158]: pgmap v374: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:06:40.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:39 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:06:40.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:39 smithi146 ceph-mon[33158]: osdmap e271: 8 total, 8 up, 8 in 2024-09-11T08:06:40.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:39 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-39"}]: dispatch 2024-09-11T08:06:40.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:39 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi102-80356-54"}]: dispatch 2024-09-11T08:06:40.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:39 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-39"}]: dispatch 2024-09-11T08:06:40.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:39 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:06:40.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:39 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:06:40.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:39 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:06:41.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:40 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:06:41.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi102-80356-54"}]': finished 2024-09-11T08:06:41.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:40 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-39"}]': finished 2024-09-11T08:06:41.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:40 smithi146 ceph-mon[33158]: osdmap e272: 8 total, 8 up, 8 in 2024-09-11T08:06:41.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi102-80356-54"}]: dispatch 2024-09-11T08:06:41.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-39", "mode": "writeback"}]: dispatch 2024-09-11T08:06:41.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:40 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-39", "mode": "writeback"}]: dispatch 2024-09-11T08:06:41.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:40 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:41.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:40 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:41.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:40 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:06:41.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:40 smithi146 ceph-mon[33158]: pgmap v377: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:41.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:40 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:06:41.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi102-80356-54"}]': finished 2024-09-11T08:06:41.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:40 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-39", "mode": "writeback"}]': finished 2024-09-11T08:06:41.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:40 smithi146 ceph-mon[33158]: osdmap e273: 8 total, 8 up, 8 in 2024-09-11T08:06:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:06:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi102-80356-54"}]': finished 2024-09-11T08:06:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-39"}]': finished 2024-09-11T08:06:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[29849]: osdmap e272: 8 total, 8 up, 8 in 2024-09-11T08:06:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi102-80356-54"}]: dispatch 2024-09-11T08:06:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-39", "mode": "writeback"}]: dispatch 2024-09-11T08:06:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-39", "mode": "writeback"}]: dispatch 2024-09-11T08:06:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:41.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:06:41.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[29849]: pgmap v377: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:41.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:06:41.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi102-80356-54"}]': finished 2024-09-11T08:06:41.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-39", "mode": "writeback"}]': finished 2024-09-11T08:06:41.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[29849]: osdmap e273: 8 total, 8 up, 8 in 2024-09-11T08:06:41.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:06:41.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi102-80356-54"}]': finished 2024-09-11T08:06:41.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-39"}]': finished 2024-09-11T08:06:41.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[41371]: osdmap e272: 8 total, 8 up, 8 in 2024-09-11T08:06:41.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi102-80356-54"}]: dispatch 2024-09-11T08:06:41.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-39", "mode": "writeback"}]: dispatch 2024-09-11T08:06:41.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-39", "mode": "writeback"}]: dispatch 2024-09-11T08:06:41.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:06:41.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:06:41.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:06:41.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[41371]: pgmap v377: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:41.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:06:41.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2557456766' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi102-80356-54"}]': finished 2024-09-11T08:06:41.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-39", "mode": "writeback"}]': finished 2024-09-11T08:06:41.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:40 smithi102 ceph-mon[41371]: osdmap e273: 8 total, 8 up, 8 in 2024-09-11T08:06:42.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi102-80356-55"}]: dispatch 2024-09-11T08:06:42.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi102-80356-55"}]: dispatch 2024-09-11T08:06:42.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi102-80356-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:42.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:42.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:41 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:42.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi102-80356-55"}]: dispatch 2024-09-11T08:06:42.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi102-80356-55"}]: dispatch 2024-09-11T08:06:42.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi102-80356-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:42.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:42.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:41 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:42.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi102-80356-55"}]: dispatch 2024-09-11T08:06:42.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi102-80356-55"}]: dispatch 2024-09-11T08:06:42.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi102-80356-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:42.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:42.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:41 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:43.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:43 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi102-80356-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:43.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:43 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:06:43.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:43 smithi102 ceph-mon[29849]: osdmap e274: 8 total, 8 up, 8 in 2024-09-11T08:06:43.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:43 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-39"}]: dispatch 2024-09-11T08:06:43.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:43 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi102-80356-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi102-80356-55"}]: dispatch 2024-09-11T08:06:43.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:43 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-39"}]: dispatch 2024-09-11T08:06:43.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:43 smithi102 ceph-mon[29849]: pgmap v380: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:43.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:43 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:06:43.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:43 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi102-80356-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:43.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:43 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:06:43.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:43 smithi102 ceph-mon[41371]: osdmap e274: 8 total, 8 up, 8 in 2024-09-11T08:06:43.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:43 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-39"}]: dispatch 2024-09-11T08:06:43.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:43 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi102-80356-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi102-80356-55"}]: dispatch 2024-09-11T08:06:43.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:43 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-39"}]: dispatch 2024-09-11T08:06:43.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:43 smithi102 ceph-mon[41371]: pgmap v380: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:43.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:43 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:06:43.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:43 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi102-80356-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:43.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:43 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:06:43.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:43 smithi146 ceph-mon[33158]: osdmap e274: 8 total, 8 up, 8 in 2024-09-11T08:06:43.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:43 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-39"}]: dispatch 2024-09-11T08:06:43.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:43 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi102-80356-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi102-80356-55"}]: dispatch 2024-09-11T08:06:43.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:43 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-39"}]: dispatch 2024-09-11T08:06:43.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:43 smithi146 ceph-mon[33158]: pgmap v380: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:43.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:43 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:06:43.943 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-09-11T08:06:43.943 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-09-11T08:06:43.943 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-09-11T08:06:43.943 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: seed 80667 2024-09-11T08:06:43.943 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-09-11T08:06:43.943 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (291 ms) 2024-09-11T08:06:43.943 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-09-11T08:06:43.944 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (12583 ms) 2024-09-11T08:06:43.944 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-09-11T08:06:43.944 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (31 ms) 2024-09-11T08:06:43.944 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (12905 ms total) 2024-09-11T08:06:43.944 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: 2024-09-11T08:06:43.944 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-09-11T08:06:43.944 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-09-11T08:06:43.944 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (8120 ms) 2024-09-11T08:06:43.944 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-09-11T08:06:43.944 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (7898 ms) 2024-09-11T08:06:43.944 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-09-11T08:06:43.944 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (9992 ms) 2024-09-11T08:06:43.944 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-09-11T08:06:43.944 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: my_snaps [4] 2024-09-11T08:06:43.945 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: my_snaps [5,4] 2024-09-11T08:06:43.945 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: my_snaps [6,5,4] 2024-09-11T08:06:43.945 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: my_snaps [7,6,5,4] 2024-09-11T08:06:43.945 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: promoting some heads 2024-09-11T08:06:43.945 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: promoting from clones for snap 7 2024-09-11T08:06:43.945 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: promoting from clones for snap 6 2024-09-11T08:06:43.945 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: promoting from clones for snap 5 2024-09-11T08:06:43.945 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: promoting from clones for snap 4 2024-09-11T08:06:43.945 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: waiting for scrubs... 2024-09-11T08:06:43.945 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: done waiting 2024-09-11T08:06:43.945 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (45996 ms) 2024-09-11T08:06:43.945 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-09-11T08:06:43.946 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (9995 ms) 2024-09-11T08:06:43.946 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-09-11T08:06:43.946 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (8044 ms) 2024-09-11T08:06:43.946 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-09-11T08:06:43.946 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8050 ms) 2024-09-11T08:06:43.946 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-09-11T08:06:43.946 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8040 ms) 2024-09-11T08:06:43.946 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-09-11T08:06:43.946 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10040 ms) 2024-09-11T08:06:43.946 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-09-11T08:06:43.946 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8842 ms) 2024-09-11T08:06:43.946 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-09-11T08:06:43.946 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10038 ms) 2024-09-11T08:06:43.947 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-09-11T08:06:43.947 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (12981 ms) 2024-09-11T08:06:43.947 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-09-11T08:06:43.947 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (8012 ms) 2024-09-11T08:06:43.947 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-09-11T08:06:43.947 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (8276 ms) 2024-09-11T08:06:43.947 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-09-11T08:06:43.947 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (12814 ms) 2024-09-11T08:06:43.947 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-09-11T08:06:43.947 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (8142 ms) 2024-09-11T08:06:43.947 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-09-11T08:06:43.947 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (7779 ms) 2024-09-11T08:06:44.588 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:44 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-39"}]': finished 2024-09-11T08:06:44.588 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:44 smithi146 ceph-mon[33158]: osdmap e275: 8 total, 8 up, 8 in 2024-09-11T08:06:44.589 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:44 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi102-80356-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi102-80356-55"}]': finished 2024-09-11T08:06:44.589 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:44 smithi146 ceph-mon[33158]: osdmap e276: 8 total, 8 up, 8 in 2024-09-11T08:06:44.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:44 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-39"}]': finished 2024-09-11T08:06:44.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:44 smithi102 ceph-mon[41371]: osdmap e275: 8 total, 8 up, 8 in 2024-09-11T08:06:44.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:44 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi102-80356-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi102-80356-55"}]': finished 2024-09-11T08:06:44.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:44 smithi102 ceph-mon[41371]: osdmap e276: 8 total, 8 up, 8 in 2024-09-11T08:06:44.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:44 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-39"}]': finished 2024-09-11T08:06:44.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:44 smithi102 ceph-mon[29849]: osdmap e275: 8 total, 8 up, 8 in 2024-09-11T08:06:44.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:44 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi102-80356-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi102-80356-55"}]': finished 2024-09-11T08:06:44.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:44 smithi102 ceph-mon[29849]: osdmap e276: 8 total, 8 up, 8 in 2024-09-11T08:06:44.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:06:44 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:06:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:06:45.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:45 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:06:45.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:45 smithi102 ceph-mon[29849]: pgmap v383: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:45.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:45 smithi102 ceph-mon[29849]: osdmap e277: 8 total, 8 up, 8 in 2024-09-11T08:06:45.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:45 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:45.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:45 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:45.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:45 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:06:45.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:45 smithi102 ceph-mon[41371]: pgmap v383: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:45.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:45 smithi102 ceph-mon[41371]: osdmap e277: 8 total, 8 up, 8 in 2024-09-11T08:06:45.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:45 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:45.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:45 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:45.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:45 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:06:45.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:45 smithi146 ceph-mon[33158]: pgmap v383: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:45.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:45 smithi146 ceph-mon[33158]: osdmap e277: 8 total, 8 up, 8 in 2024-09-11T08:06:45.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:45 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:45.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:45 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:46 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:46 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:46 smithi102 ceph-mon[41371]: osdmap e278: 8 total, 8 up, 8 in 2024-09-11T08:06:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:46 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:46 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:46 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi102-80356-55"}]: dispatch 2024-09-11T08:06:46.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:46 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:46.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:46 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:46.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:46 smithi102 ceph-mon[29849]: osdmap e278: 8 total, 8 up, 8 in 2024-09-11T08:06:46.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:46.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:46 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:46.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi102-80356-55"}]: dispatch 2024-09-11T08:06:46.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:46 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:46.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:46 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:46.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:46 smithi146 ceph-mon[33158]: osdmap e278: 8 total, 8 up, 8 in 2024-09-11T08:06:46.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:46 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:46.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:46 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:46.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:46 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi102-80356-55"}]: dispatch 2024-09-11T08:06:47.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:47 smithi102 ceph-mon[41371]: pgmap v386: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-11T08:06:47.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:47 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:06:47.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi102-80356-55"}]': finished 2024-09-11T08:06:47.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:47 smithi102 ceph-mon[41371]: osdmap e279: 8 total, 8 up, 8 in 2024-09-11T08:06:47.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi102-80356-55"}]: dispatch 2024-09-11T08:06:47.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:06:47.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:47 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:06:47.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:47 smithi102 ceph-mon[29849]: pgmap v386: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-11T08:06:47.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:47 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:06:47.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi102-80356-55"}]': finished 2024-09-11T08:06:47.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:47 smithi102 ceph-mon[29849]: osdmap e279: 8 total, 8 up, 8 in 2024-09-11T08:06:47.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi102-80356-55"}]: dispatch 2024-09-11T08:06:47.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:06:47.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:47 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:06:47.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:47 smithi146 ceph-mon[33158]: pgmap v386: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-11T08:06:47.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:47 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:06:47.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi102-80356-55"}]': finished 2024-09-11T08:06:47.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:47 smithi146 ceph-mon[33158]: osdmap e279: 8 total, 8 up, 8 in 2024-09-11T08:06:47.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi102-80356-55"}]: dispatch 2024-09-11T08:06:47.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:06:47.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:47 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:06:49.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi102-80356-55"}]': finished 2024-09-11T08:06:49.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:48 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:06:49.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:48 smithi102 ceph-mon[41371]: osdmap e280: 8 total, 8 up, 8 in 2024-09-11T08:06:49.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:06:49.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:48 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:06:49.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi102-80356-56"}]: dispatch 2024-09-11T08:06:49.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi102-80356-56"}]: dispatch 2024-09-11T08:06:49.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi102-80356-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:49.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:48 smithi102 ceph-mon[41371]: pgmap v389: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-11T08:06:49.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi102-80356-55"}]': finished 2024-09-11T08:06:49.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:48 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:06:49.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:48 smithi102 ceph-mon[29849]: osdmap e280: 8 total, 8 up, 8 in 2024-09-11T08:06:49.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:06:49.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:48 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:06:49.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi102-80356-56"}]: dispatch 2024-09-11T08:06:49.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi102-80356-56"}]: dispatch 2024-09-11T08:06:49.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi102-80356-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:49.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:48 smithi102 ceph-mon[29849]: pgmap v389: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-11T08:06:49.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3193739302' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi102-80356-55"}]': finished 2024-09-11T08:06:49.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:48 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:06:49.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:48 smithi146 ceph-mon[33158]: osdmap e280: 8 total, 8 up, 8 in 2024-09-11T08:06:49.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:06:49.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:48 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:06:49.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi102-80356-56"}]: dispatch 2024-09-11T08:06:49.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi102-80356-56"}]: dispatch 2024-09-11T08:06:49.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi102-80356-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:49.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:48 smithi146 ceph-mon[33158]: pgmap v389: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-11T08:06:49.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:06:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:06:50.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:06:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:06:49] "GET /metrics HTTP/1.1" 200 228507 "" "Prometheus/2.43.0" 2024-09-11T08:06:50.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:50 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:06:50.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:50 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi102-80356-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:50.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:50 smithi102 ceph-mon[29849]: osdmap e281: 8 total, 8 up, 8 in 2024-09-11T08:06:50.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:50 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:06:50.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:50 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi102-80356-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi102-80356-56"}]: dispatch 2024-09-11T08:06:50.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:50 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:06:50.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:50 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:06:50.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:50 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:06:50.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:50 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi102-80356-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:50.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:50 smithi102 ceph-mon[41371]: osdmap e281: 8 total, 8 up, 8 in 2024-09-11T08:06:50.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:50 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:06:50.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:50 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi102-80356-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi102-80356-56"}]: dispatch 2024-09-11T08:06:50.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:50 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:06:50.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:50 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:06:50.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:50 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:06:50.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:50 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi102-80356-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:50.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:50 smithi146 ceph-mon[33158]: osdmap e281: 8 total, 8 up, 8 in 2024-09-11T08:06:50.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:50 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:06:50.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:50 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi102-80356-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi102-80356-56"}]: dispatch 2024-09-11T08:06:50.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:50 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:06:50.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:50 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:06:51.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:51 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:06:51.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:51 smithi102 ceph-mon[29849]: osdmap e282: 8 total, 8 up, 8 in 2024-09-11T08:06:51.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:51 smithi102 ceph-mon[29849]: pgmap v392: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:51.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:51 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi102-80356-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi102-80356-56"}]': finished 2024-09-11T08:06:51.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:51 smithi102 ceph-mon[29849]: osdmap e283: 8 total, 8 up, 8 in 2024-09-11T08:06:51.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:51 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:06:51.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:51 smithi102 ceph-mon[41371]: osdmap e282: 8 total, 8 up, 8 in 2024-09-11T08:06:51.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:51 smithi102 ceph-mon[41371]: pgmap v392: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:51.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:51 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi102-80356-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi102-80356-56"}]': finished 2024-09-11T08:06:51.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:51 smithi102 ceph-mon[41371]: osdmap e283: 8 total, 8 up, 8 in 2024-09-11T08:06:51.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:51 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:06:51.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:51 smithi146 ceph-mon[33158]: osdmap e282: 8 total, 8 up, 8 in 2024-09-11T08:06:51.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:51 smithi146 ceph-mon[33158]: pgmap v392: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:51.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:51 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi102-80356-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi102-80356-56"}]': finished 2024-09-11T08:06:51.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:51 smithi146 ceph-mon[33158]: osdmap e283: 8 total, 8 up, 8 in 2024-09-11T08:06:52.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:52 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:52.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:52 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:52.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:52 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-41"}]: dispatch 2024-09-11T08:06:52.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:52 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-41"}]: dispatch 2024-09-11T08:06:52.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:52 smithi102 ceph-mon[29849]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:52.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:52 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-41"}]': finished 2024-09-11T08:06:52.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:52 smithi102 ceph-mon[29849]: osdmap e284: 8 total, 8 up, 8 in 2024-09-11T08:06:52.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:52 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:52.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:52 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:52.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:52 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-41"}]: dispatch 2024-09-11T08:06:52.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:52 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-41"}]: dispatch 2024-09-11T08:06:52.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:52 smithi102 ceph-mon[41371]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:52.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:52 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-41"}]': finished 2024-09-11T08:06:52.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:52 smithi102 ceph-mon[41371]: osdmap e284: 8 total, 8 up, 8 in 2024-09-11T08:06:52.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:52 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:52.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:52 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:06:52.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:52 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-41"}]: dispatch 2024-09-11T08:06:52.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:52 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-41"}]: dispatch 2024-09-11T08:06:52.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:52 smithi146 ceph-mon[33158]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:52.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:52 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-41"}]': finished 2024-09-11T08:06:52.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:52 smithi146 ceph-mon[33158]: osdmap e284: 8 total, 8 up, 8 in 2024-09-11T08:06:53.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:53 smithi102 ceph-mon[29849]: pgmap v395: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:53.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:53 smithi102 ceph-mon[29849]: osdmap e285: 8 total, 8 up, 8 in 2024-09-11T08:06:53.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:53 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi102-80356-56"}]: dispatch 2024-09-11T08:06:53.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:53 smithi102 ceph-mon[41371]: pgmap v395: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:53.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:53 smithi102 ceph-mon[41371]: osdmap e285: 8 total, 8 up, 8 in 2024-09-11T08:06:53.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:53 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi102-80356-56"}]: dispatch 2024-09-11T08:06:53.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:53 smithi146 ceph-mon[33158]: pgmap v395: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:53.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:53 smithi146 ceph-mon[33158]: osdmap e285: 8 total, 8 up, 8 in 2024-09-11T08:06:53.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:53 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi102-80356-56"}]: dispatch 2024-09-11T08:06:54.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:06:54 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:06:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:06:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:54 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi102-80356-56"}]': finished 2024-09-11T08:06:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:54 smithi102 ceph-mon[29849]: osdmap e286: 8 total, 8 up, 8 in 2024-09-11T08:06:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:54 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi102-80356-56"}]: dispatch 2024-09-11T08:06:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:54 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:54 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:54 smithi102 ceph-mon[29849]: pgmap v398: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:55.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:54 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi102-80356-56"}]': finished 2024-09-11T08:06:55.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:54 smithi102 ceph-mon[41371]: osdmap e286: 8 total, 8 up, 8 in 2024-09-11T08:06:55.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:54 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi102-80356-56"}]: dispatch 2024-09-11T08:06:55.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:54 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:55.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:54 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:55.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:54 smithi102 ceph-mon[41371]: pgmap v398: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:54 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi102-80356-56"}]': finished 2024-09-11T08:06:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:54 smithi146 ceph-mon[33158]: osdmap e286: 8 total, 8 up, 8 in 2024-09-11T08:06:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:54 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi102-80356-56"}]: dispatch 2024-09-11T08:06:55.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:54 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:55.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:54 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:06:55.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:54 smithi146 ceph-mon[33158]: pgmap v398: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:06:56.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi102-80356-56"}]': finished 2024-09-11T08:06:56.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:56.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[41371]: osdmap e287: 8 total, 8 up, 8 in 2024-09-11T08:06:56.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi102-80667-6","var": "pg_num","format": "json"}]: dispatch 2024-09-11T08:06:56.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:56.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:56.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3566180117' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:06:56.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[41371]: from='client.46982 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:06:56.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3566180117' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:06:56.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[41371]: from='client.46982 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:06:56.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3566180117' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi102-80356-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:56.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[41371]: from='client.46982 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi102-80356-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:56.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi102-80356-56"}]': finished 2024-09-11T08:06:56.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:56.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[29849]: osdmap e287: 8 total, 8 up, 8 in 2024-09-11T08:06:56.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi102-80667-6","var": "pg_num","format": "json"}]: dispatch 2024-09-11T08:06:56.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:56.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:56.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3566180117' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:06:56.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[29849]: from='client.46982 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:06:56.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3566180117' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:06:56.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[29849]: from='client.46982 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:06:56.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3566180117' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi102-80356-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:56.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:55 smithi102 ceph-mon[29849]: from='client.46982 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi102-80356-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:56.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3219822219' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi102-80356-56"}]': finished 2024-09-11T08:06:56.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:55 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:06:56.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:55 smithi146 ceph-mon[33158]: osdmap e287: 8 total, 8 up, 8 in 2024-09-11T08:06:56.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi102-80667-6","var": "pg_num","format": "json"}]: dispatch 2024-09-11T08:06:56.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:56.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:55 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:06:56.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3566180117' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:06:56.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:55 smithi146 ceph-mon[33158]: from='client.46982 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:06:56.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3566180117' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:06:56.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:55 smithi146 ceph-mon[33158]: from='client.46982 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:06:56.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3566180117' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi102-80356-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:56.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:55 smithi146 ceph-mon[33158]: from='client.46982 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi102-80356-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:06:57.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:57 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:06:57.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:57 smithi102 ceph-mon[29849]: from='client.46982 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi102-80356-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:57.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:57 smithi102 ceph-mon[29849]: osdmap e288: 8 total, 8 up, 8 in 2024-09-11T08:06:57.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:57 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-11T08:06:57.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:57 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-11T08:06:57.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:57 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3566180117' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi102-80356-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:06:57.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:57 smithi102 ceph-mon[29849]: from='client.46982 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi102-80356-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:06:57.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:57 smithi102 ceph-mon[29849]: pgmap v401: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:57.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:57 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:57.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:57 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:06:57.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:57 smithi102 ceph-mon[41371]: from='client.46982 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi102-80356-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:57.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:57 smithi102 ceph-mon[41371]: osdmap e288: 8 total, 8 up, 8 in 2024-09-11T08:06:57.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:57 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-11T08:06:57.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:57 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-11T08:06:57.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:57 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3566180117' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi102-80356-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:06:57.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:57 smithi102 ceph-mon[41371]: from='client.46982 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi102-80356-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:06:57.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:57 smithi102 ceph-mon[41371]: pgmap v401: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:57.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:57 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:57.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:57 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:06:57.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:57 smithi146 ceph-mon[33158]: from='client.46982 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi102-80356-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:06:57.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:57 smithi146 ceph-mon[33158]: osdmap e288: 8 total, 8 up, 8 in 2024-09-11T08:06:57.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:57 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-11T08:06:57.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:57 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-11T08:06:57.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:57 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3566180117' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi102-80356-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:06:57.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:57 smithi146 ceph-mon[33158]: from='client.46982 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi102-80356-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:06:57.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:57 smithi146 ceph-mon[33158]: pgmap v401: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:57.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:57 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:06:58.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:58 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_count","val": "8"}]': finished 2024-09-11T08:06:58.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:58 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:06:58.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:58 smithi102 ceph-mon[41371]: osdmap e289: 8 total, 8 up, 8 in 2024-09-11T08:06:58.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:58 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:06:58.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:58 smithi102 ceph-mon[41371]: from='client.46982 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi102-80356-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi102-80356-57"}]': finished 2024-09-11T08:06:58.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:58 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:06:58.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:58 smithi102 ceph-mon[41371]: osdmap e290: 8 total, 8 up, 8 in 2024-09-11T08:06:58.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:58 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-11T08:06:58.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:58 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-11T08:06:58.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:58 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_count","val": "8"}]': finished 2024-09-11T08:06:58.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:58 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:06:58.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:58 smithi102 ceph-mon[29849]: osdmap e289: 8 total, 8 up, 8 in 2024-09-11T08:06:58.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:58 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:06:58.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:58 smithi102 ceph-mon[29849]: from='client.46982 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi102-80356-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi102-80356-57"}]': finished 2024-09-11T08:06:58.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:58 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:06:58.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:58 smithi102 ceph-mon[29849]: osdmap e290: 8 total, 8 up, 8 in 2024-09-11T08:06:58.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:58 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-11T08:06:58.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:58 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-11T08:06:58.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:58 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_count","val": "8"}]': finished 2024-09-11T08:06:58.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:58 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:06:58.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:58 smithi146 ceph-mon[33158]: osdmap e289: 8 total, 8 up, 8 in 2024-09-11T08:06:58.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:58 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:06:58.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:58 smithi146 ceph-mon[33158]: from='client.46982 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi102-80356-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi102-80356-57"}]': finished 2024-09-11T08:06:58.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:58 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:06:58.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:58 smithi146 ceph-mon[33158]: osdmap e290: 8 total, 8 up, 8 in 2024-09-11T08:06:58.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:58 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-11T08:06:58.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:58 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-11T08:06:59.187 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-09-11T08:06:59.187 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: hmm, no HitSet yet 2024-09-11T08:06:59.187 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: ok, hit_set contains 193:602f83fe:::foo:head 2024-09-11T08:06:59.187 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9022 ms) 2024-09-11T08:06:59.187 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-09-11T08:06:59.188 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg_num = 32 2024-09-11T08:06:59.188 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 0 ls 1726042019,0 2024-09-11T08:06:59.188 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 1 ls 1726042019,0 2024-09-11T08:06:59.188 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 2 ls 1726042019,0 2024-09-11T08:06:59.188 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 3 ls 1726042019,0 2024-09-11T08:06:59.188 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 4 ls 1726042019,0 2024-09-11T08:06:59.188 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 5 ls 1726042019,0 2024-09-11T08:06:59.188 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 6 ls 1726042019,0 2024-09-11T08:06:59.188 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 7 ls 1726042019,0 2024-09-11T08:06:59.188 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 8 ls 1726042019,0 2024-09-11T08:06:59.188 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 9 ls 1726042019,0 2024-09-11T08:06:59.188 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 10 ls 1726042019,0 2024-09-11T08:06:59.189 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 11 ls 1726042019,0 2024-09-11T08:06:59.189 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 12 ls 1726042019,0 2024-09-11T08:06:59.189 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 13 ls 1726042019,0 2024-09-11T08:06:59.189 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 14 ls 1726042019,0 2024-09-11T08:06:59.189 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 15 ls 1726042019,0 2024-09-11T08:06:59.189 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 16 ls 1726042019,0 2024-09-11T08:06:59.189 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 17 ls 1726042019,0 2024-09-11T08:06:59.189 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 18 ls 1726042019,0 2024-09-11T08:06:59.189 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 19 ls 1726042019,0 2024-09-11T08:06:59.189 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 20 ls 1726042019,0 2024-09-11T08:06:59.189 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 21 ls 1726042019,0 2024-09-11T08:06:59.189 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 22 ls 1726042019,0 2024-09-11T08:06:59.190 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 23 ls 1726042019,0 2024-09-11T08:06:59.190 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 24 ls 1726042019,0 2024-09-11T08:06:59.190 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 25 ls 1726042019,0 2024-09-11T08:06:59.190 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 26 ls 1726042019,0 2024-09-11T08:06:59.190 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 27 ls 1726042019,0 2024-09-11T08:06:59.190 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 28 ls 1726042019,0 2024-09-11T08:06:59.190 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 29 ls 1726042019,0 2024-09-11T08:06:59.190 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 30 ls 1726042019,0 2024-09-11T08:06:59.190 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg 31 ls 1726042019,0 2024-09-11T08:06:59.190 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: pg_num = 32 2024-09-11T08:06:59.190 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:6cac518f:::0:head 2024-09-11T08:06:59.190 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:02547ec2:::1:head 2024-09-11T08:06:59.191 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:f905c69b:::2:head 2024-09-11T08:06:59.191 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:cfc208b3:::3:head 2024-09-11T08:06:59.191 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:d83876eb:::4:head 2024-09-11T08:06:59.191 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:b29083e3:::5:head 2024-09-11T08:06:59.191 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:c4fdafeb:::6:head 2024-09-11T08:06:59.191 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:5c6b0b28:::7:head 2024-09-11T08:06:59.191 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:bd63b0f1:::8:head 2024-09-11T08:06:59.191 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:e960b815:::9:head 2024-09-11T08:06:59.191 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:52ea6a34:::10:head 2024-09-11T08:06:59.191 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:89d3ae78:::11:head 2024-09-11T08:06:59.191 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:de5d7c5f:::12:head 2024-09-11T08:06:59.191 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:566253c9:::13:head 2024-09-11T08:06:59.192 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:62a1935d:::14:head 2024-09-11T08:06:59.192 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:863748b0:::15:head 2024-09-11T08:06:59.192 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:3958e169:::16:head 2024-09-11T08:06:59.192 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:4d4dabf9:::17:head 2024-09-11T08:06:59.192 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:8391935d:::18:head 2024-09-11T08:06:59.192 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:28883081:::19:head 2024-09-11T08:06:59.192 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:69259c59:::20:head 2024-09-11T08:06:59.192 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:4bdb80b7:::21:head 2024-09-11T08:06:59.192 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:a11c5d71:::22:head 2024-09-11T08:06:59.192 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:271af37b:::23:head 2024-09-11T08:06:59.192 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:95b121be:::24:head 2024-09-11T08:06:59.192 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:58d1031b:::25:head 2024-09-11T08:06:59.193 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:0a050783:::26:head 2024-09-11T08:06:59.193 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:c709704c:::27:head 2024-09-11T08:06:59.193 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:cbe56eaf:::28:head 2024-09-11T08:06:59.193 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:86b4b162:::29:head 2024-09-11T08:06:59.193 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:70d89383:::30:head 2024-09-11T08:06:59.193 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:dd450c7c:::31:head 2024-09-11T08:06:59.193 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:6d5729b1:::32:head 2024-09-11T08:06:59.193 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:c388f3fb:::33:head 2024-09-11T08:06:59.193 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:56cfea31:::34:head 2024-09-11T08:06:59.193 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:9dbc1bf7:::35:head 2024-09-11T08:06:59.193 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:40b74ccd:::36:head 2024-09-11T08:06:59.193 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:4d5aaf42:::37:head 2024-09-11T08:06:59.193 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:920f362c:::38:head 2024-09-11T08:06:59.194 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:6cc53222:::39:head 2024-09-11T08:06:59.194 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:9cad833f:::40:head 2024-09-11T08:06:59.194 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:1ea84d41:::41:head 2024-09-11T08:06:59.194 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:c4480ef6:::42:head 2024-09-11T08:06:59.194 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:a694361e:::43:head 2024-09-11T08:06:59.194 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:d1bd33e9:::44:head 2024-09-11T08:06:59.194 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:ddc2cd5d:::45:head 2024-09-11T08:06:59.194 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:2b782207:::46:head 2024-09-11T08:06:59.194 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:7b187fca:::47:head 2024-09-11T08:06:59.194 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:90ecdf6f:::48:head 2024-09-11T08:06:59.194 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:a5ed95fe:::49:head 2024-09-11T08:06:59.194 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:ea0eaa55:::50:head 2024-09-11T08:06:59.194 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:f33ef17b:::51:head 2024-09-11T08:06:59.195 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:a0d1b2f6:::52:head 2024-09-11T08:06:59.195 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:60c5229e:::53:head 2024-09-11T08:06:59.195 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:edcbc575:::54:head 2024-09-11T08:06:59.195 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:102cf253:::55:head 2024-09-11T08:06:59.195 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:efb7fb0b:::56:head 2024-09-11T08:06:59.195 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:50d0a326:::57:head 2024-09-11T08:06:59.195 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:d4dc5daf:::58:head 2024-09-11T08:06:59.195 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:3a130462:::59:head 2024-09-11T08:06:59.195 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:ec87ed71:::60:head 2024-09-11T08:06:59.195 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:d5bc9454:::61:head 2024-09-11T08:06:59.195 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:3ddfe313:::62:head 2024-09-11T08:06:59.195 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:7c2816b9:::63:head 2024-09-11T08:06:59.195 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:47e00e4d:::64:head 2024-09-11T08:06:59.196 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:c6410c18:::65:head 2024-09-11T08:06:59.196 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:b48ed237:::66:head 2024-09-11T08:06:59.196 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:cd63ad31:::67:head 2024-09-11T08:06:59.196 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:b179e92b:::68:head 2024-09-11T08:06:59.196 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:0d9f741a:::69:head 2024-09-11T08:06:59.196 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:6d3352ae:::70:head 2024-09-11T08:06:59.196 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:c6d5c19e:::71:head 2024-09-11T08:06:59.196 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:bc4729c3:::72:head 2024-09-11T08:06:59.196 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:77e930b9:::73:head 2024-09-11T08:06:59.196 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:0abeecfd:::74:head 2024-09-11T08:06:59.196 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:b7c37e15:::75:head 2024-09-11T08:06:59.196 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:b6378398:::76:head 2024-09-11T08:06:59.197 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:02bd68de:::77:head 2024-09-11T08:06:59.197 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:cc795d2d:::78:head 2024-09-11T08:06:59.197 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:630d4fea:::79:head 2024-09-11T08:06:59.197 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:e0d29ef5:::80:head 2024-09-11T08:06:59.197 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:fd6f13d2:::81:head 2024-09-11T08:06:59.197 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:606461d5:::82:head 2024-09-11T08:06:59.197 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:eadbdc43:::83:head 2024-09-11T08:06:59.197 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:8761d0bb:::84:head 2024-09-11T08:06:59.197 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:9ef0186f:::85:head 2024-09-11T08:06:59.197 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:e0d41294:::86:head 2024-09-11T08:06:59.197 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:961de695:::87:head 2024-09-11T08:06:59.197 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:1423148f:::88:head 2024-09-11T08:06:59.197 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:633a8fa2:::89:head 2024-09-11T08:06:59.198 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:a8653809:::90:head 2024-09-11T08:06:59.198 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:3dac8b33:::91:head 2024-09-11T08:06:59.198 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:35aad435:::92:head 2024-09-11T08:06:59.198 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:f6dcc343:::93:head 2024-09-11T08:06:59.198 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:dbbdad87:::94:head 2024-09-11T08:06:59.198 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:1cb48ce0:::95:head 2024-09-11T08:06:59.198 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:03cd461c:::96:head 2024-09-11T08:06:59.198 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:17a4ea99:::97:head 2024-09-11T08:06:59.198 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:9993c9a7:::98:head 2024-09-11T08:06:59.198 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:6394211c:::99:head 2024-09-11T08:06:59.198 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:94c7ae57:::100:head 2024-09-11T08:06:59.198 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:6fdee5bb:::101:head 2024-09-11T08:06:59.198 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:9a477fd1:::102:head 2024-09-11T08:06:59.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:eb850916:::103:head 2024-09-11T08:06:59.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:affc56b9:::104:head 2024-09-11T08:06:59.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:b42dc814:::105:head 2024-09-11T08:06:59.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:f319f8f0:::106:head 2024-09-11T08:06:59.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:9a40b9de:::107:head 2024-09-11T08:06:59.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:8b524f28:::108:head 2024-09-11T08:06:59.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:e3de589f:::109:head 2024-09-11T08:06:59.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:90f90a5b:::110:head 2024-09-11T08:06:59.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:a7b4f1d7:::111:head 2024-09-11T08:06:59.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:af51766e:::112:head 2024-09-11T08:06:59.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:b6f90bd1:::113:head 2024-09-11T08:06:59.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:e0261208:::114:head 2024-09-11T08:06:59.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:c9569ef7:::115:head 2024-09-11T08:06:59.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:61bebe50:::116:head 2024-09-11T08:06:59.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:fe93412b:::117:head 2024-09-11T08:06:59.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:d3d38bee:::118:head 2024-09-11T08:06:59.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:3100ba0c:::119:head 2024-09-11T08:06:59.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:d0560ada:::120:head 2024-09-11T08:06:59.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:f0ea8b35:::121:head 2024-09-11T08:06:59.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:766f231a:::122:head 2024-09-11T08:06:59.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:a07a2582:::123:head 2024-09-11T08:06:59.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:bd7c6b3a:::124:head 2024-09-11T08:06:59.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:fb2ddaff:::125:head 2024-09-11T08:06:59.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:4408e1fe:::126:head 2024-09-11T08:06:59.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:ee1df7a7:::127:head 2024-09-11T08:06:59.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:c3002909:::128:head 2024-09-11T08:06:59.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:4f48ffa9:::129:head 2024-09-11T08:06:59.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:edf38733:::130:head 2024-09-11T08:06:59.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:c08425c0:::131:head 2024-09-11T08:06:59.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:5f902d98:::132:head 2024-09-11T08:06:59.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:41ea2c93:::133:head 2024-09-11T08:06:59.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:813cee13:::134:head 2024-09-11T08:06:59.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:0131818d:::135:head 2024-09-11T08:06:59.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:26ba5a85:::136:head 2024-09-11T08:06:59.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:381b8a5a:::137:head 2024-09-11T08:06:59.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:28797e47:::138:head 2024-09-11T08:06:59.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:bfca7f22:::139:head 2024-09-11T08:06:59.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:36807075:::140:head 2024-09-11T08:06:59.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:80b03975:::141:head 2024-09-11T08:06:59.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:5c15709b:::142:head 2024-09-11T08:06:59.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:f39ea15e:::143:head 2024-09-11T08:06:59.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:ea992956:::144:head 2024-09-11T08:06:59.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:48887b1c:::145:head 2024-09-11T08:06:59.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:9f24a9dd:::146:head 2024-09-11T08:06:59.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:987f100b:::147:head 2024-09-11T08:06:59.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:d2dd3581:::148:head 2024-09-11T08:06:59.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:7fed1808:::149:head 2024-09-11T08:06:59.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:c80b70e9:::150:head 2024-09-11T08:06:59.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:85ed90f9:::151:head 2024-09-11T08:06:59.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:36428b24:::152:head 2024-09-11T08:06:59.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:d044c34a:::153:head 2024-09-11T08:06:59.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:7c18bf58:::154:head 2024-09-11T08:06:59.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:d1c21232:::155:head 2024-09-11T08:06:59.203 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:a7a3c575:::156:head 2024-09-11T08:06:59.203 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:87da0633:::157:head 2024-09-11T08:06:59.203 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:d5ac3822:::158:head 2024-09-11T08:06:59.203 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:3f20522d:::159:head 2024-09-11T08:06:59.203 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:6ca26563:::160:head 2024-09-11T08:06:59.203 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:532ce135:::161:head 2024-09-11T08:06:59.203 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:c78863e6:::162:head 2024-09-11T08:06:59.641 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:59 smithi146 ceph-mon[33158]: pgmap v404: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:59.641 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:59 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-11T08:06:59.641 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:59 smithi146 ceph-mon[33158]: osdmap e291: 8 total, 8 up, 8 in 2024-09-11T08:06:59.642 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:06:59 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi102-80667-43","var": "pg_num","format": "json"}]: dispatch 2024-09-11T08:06:59.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:59 smithi102 ceph-mon[41371]: pgmap v404: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:59.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:59 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-11T08:06:59.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:59 smithi102 ceph-mon[41371]: osdmap e291: 8 total, 8 up, 8 in 2024-09-11T08:06:59.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:06:59 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi102-80667-43","var": "pg_num","format": "json"}]: dispatch 2024-09-11T08:06:59.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:59 smithi102 ceph-mon[29849]: pgmap v404: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:06:59.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:59 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-11T08:06:59.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:59 smithi102 ceph-mon[29849]: osdmap e291: 8 total, 8 up, 8 in 2024-09-11T08:06:59.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:06:59 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi102-80667-43","var": "pg_num","format": "json"}]: dispatch 2024-09-11T08:06:59.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:06:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:07:00.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:06:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:06:59] "GET /metrics HTTP/1.1" 200 230924 "" "Prometheus/2.43.0" 2024-09-11T08:07:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:00 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:00 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:00 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-43"}]: dispatch 2024-09-11T08:07:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:00 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-43"}]: dispatch 2024-09-11T08:07:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:00 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:07:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:00 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:00 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-43"}]': finished 2024-09-11T08:07:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:00 smithi102 ceph-mon[41371]: osdmap e292: 8 total, 8 up, 8 in 2024-09-11T08:07:00.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:00 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3566180117' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:07:00.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:00 smithi102 ceph-mon[41371]: from='client.46982 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:07:00.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:00 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:00.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:00 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:00.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:00 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-43"}]: dispatch 2024-09-11T08:07:00.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:00 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-43"}]: dispatch 2024-09-11T08:07:00.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:00 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:07:00.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:00 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:00.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:00 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-43"}]': finished 2024-09-11T08:07:00.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:00 smithi102 ceph-mon[29849]: osdmap e292: 8 total, 8 up, 8 in 2024-09-11T08:07:00.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:00 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3566180117' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:07:00.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:00 smithi102 ceph-mon[29849]: from='client.46982 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:07:00.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:00 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:00.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:00 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:00.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:00 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-43"}]: dispatch 2024-09-11T08:07:00.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:00 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-43"}]: dispatch 2024-09-11T08:07:00.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:00 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:07:00.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:00 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:00.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:00 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-43"}]': finished 2024-09-11T08:07:00.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:00 smithi146 ceph-mon[33158]: osdmap e292: 8 total, 8 up, 8 in 2024-09-11T08:07:00.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:00 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3566180117' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:07:00.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:00 smithi146 ceph-mon[33158]: from='client.46982 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:07:01.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:01 smithi102 ceph-mon[41371]: pgmap v407: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 KiB/s wr, 101 op/s 2024-09-11T08:07:01.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:01 smithi102 ceph-mon[41371]: from='client.46982 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi102-80356-57"}]': finished 2024-09-11T08:07:01.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:01 smithi102 ceph-mon[41371]: osdmap e293: 8 total, 8 up, 8 in 2024-09-11T08:07:01.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:01 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3566180117' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:07:01.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:01 smithi102 ceph-mon[41371]: from='client.46982 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:07:01.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:01 smithi102 ceph-mon[29849]: pgmap v407: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 KiB/s wr, 101 op/s 2024-09-11T08:07:01.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:01 smithi102 ceph-mon[29849]: from='client.46982 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi102-80356-57"}]': finished 2024-09-11T08:07:01.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:01 smithi102 ceph-mon[29849]: osdmap e293: 8 total, 8 up, 8 in 2024-09-11T08:07:01.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:01 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3566180117' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:07:01.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:01 smithi102 ceph-mon[29849]: from='client.46982 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:07:01.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:01 smithi146 ceph-mon[33158]: pgmap v407: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 KiB/s wr, 101 op/s 2024-09-11T08:07:01.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:01 smithi146 ceph-mon[33158]: from='client.46982 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi102-80356-57"}]': finished 2024-09-11T08:07:01.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:01 smithi146 ceph-mon[33158]: osdmap e293: 8 total, 8 up, 8 in 2024-09-11T08:07:01.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:01 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3566180117' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:07:01.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:01 smithi146 ceph-mon[33158]: from='client.46982 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi102-80356-57"}]: dispatch 2024-09-11T08:07:02.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:02 smithi102 ceph-mon[41371]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:02.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:02 smithi102 ceph-mon[41371]: from='client.46982 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi102-80356-57"}]': finished 2024-09-11T08:07:02.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:02 smithi102 ceph-mon[41371]: osdmap e294: 8 total, 8 up, 8 in 2024-09-11T08:07:02.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:02 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:02.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:02 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:02.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:02 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2989381855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:02.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:02 smithi102 ceph-mon[41371]: from='client.46988 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:02.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:02 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2989381855' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:02.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:02 smithi102 ceph-mon[41371]: from='client.46988 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:02.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:02 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2989381855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi102-80356-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:02.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:02 smithi102 ceph-mon[41371]: from='client.46988 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi102-80356-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:02.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:02 smithi102 ceph-mon[29849]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:02.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:02 smithi102 ceph-mon[29849]: from='client.46982 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi102-80356-57"}]': finished 2024-09-11T08:07:02.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:02 smithi102 ceph-mon[29849]: osdmap e294: 8 total, 8 up, 8 in 2024-09-11T08:07:02.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:02 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:02.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:02 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:02.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:02 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2989381855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:02.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:02 smithi102 ceph-mon[29849]: from='client.46988 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:02.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:02 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2989381855' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:02.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:02 smithi102 ceph-mon[29849]: from='client.46988 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:02.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:02 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2989381855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi102-80356-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:02.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:02 smithi102 ceph-mon[29849]: from='client.46988 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi102-80356-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:02.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:02 smithi146 ceph-mon[33158]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:02.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:02 smithi146 ceph-mon[33158]: from='client.46982 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi102-80356-57"}]': finished 2024-09-11T08:07:02.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:02 smithi146 ceph-mon[33158]: osdmap e294: 8 total, 8 up, 8 in 2024-09-11T08:07:02.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:02 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:02.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:02 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:02.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:02 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2989381855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:02.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:02 smithi146 ceph-mon[33158]: from='client.46988 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:02.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:02 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2989381855' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:02.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:02 smithi146 ceph-mon[33158]: from='client.46988 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:02.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:02 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2989381855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi102-80356-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:02.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:02 smithi146 ceph-mon[33158]: from='client.46988 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi102-80356-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:03 smithi102 ceph-mon[41371]: pgmap v410: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:03 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:07:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:03 smithi102 ceph-mon[41371]: from='client.46988 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi102-80356-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:03 smithi102 ceph-mon[41371]: osdmap e295: 8 total, 8 up, 8 in 2024-09-11T08:07:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:03 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2989381855' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi102-80356-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:03 smithi102 ceph-mon[41371]: from='client.46988 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi102-80356-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:03 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:03 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:03.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:03 smithi102 ceph-mon[29849]: pgmap v410: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:03.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:03 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:07:03.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:03 smithi102 ceph-mon[29849]: from='client.46988 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi102-80356-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:03.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:03 smithi102 ceph-mon[29849]: osdmap e295: 8 total, 8 up, 8 in 2024-09-11T08:07:03.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:03 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2989381855' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi102-80356-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:03.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:03 smithi102 ceph-mon[29849]: from='client.46988 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi102-80356-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:03.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:03 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:03.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:03 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:03 smithi146 ceph-mon[33158]: pgmap v410: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:03 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:07:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:03 smithi146 ceph-mon[33158]: from='client.46988 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi102-80356-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:03 smithi146 ceph-mon[33158]: osdmap e295: 8 total, 8 up, 8 in 2024-09-11T08:07:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:03 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2989381855' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi102-80356-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:03 smithi146 ceph-mon[33158]: from='client.46988 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi102-80356-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:03 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:03 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:04.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:07:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:07:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:07:05.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:04 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:07:05.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:04 smithi102 ceph-mon[41371]: osdmap e296: 8 total, 8 up, 8 in 2024-09-11T08:07:05.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:04 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-11T08:07:05.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:04 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-11T08:07:05.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:04 smithi102 ceph-mon[41371]: pgmap v413: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:04 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:07:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:04 smithi102 ceph-mon[29849]: osdmap e296: 8 total, 8 up, 8 in 2024-09-11T08:07:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:04 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-11T08:07:05.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:04 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-11T08:07:05.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:04 smithi102 ceph-mon[29849]: pgmap v413: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:04 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:07:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:04 smithi146 ceph-mon[33158]: osdmap e296: 8 total, 8 up, 8 in 2024-09-11T08:07:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:04 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-11T08:07:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:04 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-11T08:07:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:04 smithi146 ceph-mon[33158]: pgmap v413: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:06.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:05 smithi102 ceph-mon[41371]: from='client.46988 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi102-80356-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi102-80356-58"}]': finished 2024-09-11T08:07:06.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:05 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_count","val": "3"}]': finished 2024-09-11T08:07:06.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:05 smithi102 ceph-mon[41371]: osdmap e297: 8 total, 8 up, 8 in 2024-09-11T08:07:06.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:05 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-11T08:07:06.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:05 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-11T08:07:06.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:05 smithi102 ceph-mon[29849]: from='client.46988 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi102-80356-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi102-80356-58"}]': finished 2024-09-11T08:07:06.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:05 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_count","val": "3"}]': finished 2024-09-11T08:07:06.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:05 smithi102 ceph-mon[29849]: osdmap e297: 8 total, 8 up, 8 in 2024-09-11T08:07:06.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:05 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-11T08:07:06.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:05 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-11T08:07:06.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:05 smithi146 ceph-mon[33158]: from='client.46988 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi102-80356-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi102-80356-58"}]': finished 2024-09-11T08:07:06.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:05 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_count","val": "3"}]': finished 2024-09-11T08:07:06.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:05 smithi146 ceph-mon[33158]: osdmap e297: 8 total, 8 up, 8 in 2024-09-11T08:07:06.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:05 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-11T08:07:06.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:05 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-11T08:07:07.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:07 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_period","val": "3"}]': finished 2024-09-11T08:07:07.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:07 smithi102 ceph-mon[41371]: osdmap e298: 8 total, 8 up, 8 in 2024-09-11T08:07:07.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:07:07.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:07 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:07:07.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:07 smithi102 ceph-mon[41371]: pgmap v416: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:07 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_period","val": "3"}]': finished 2024-09-11T08:07:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:07 smithi102 ceph-mon[29849]: osdmap e298: 8 total, 8 up, 8 in 2024-09-11T08:07:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:07:07.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:07 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:07:07.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:07 smithi102 ceph-mon[29849]: pgmap v416: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:07 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_period","val": "3"}]': finished 2024-09-11T08:07:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:07 smithi146 ceph-mon[33158]: osdmap e298: 8 total, 8 up, 8 in 2024-09-11T08:07:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:07:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:07 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:07:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:07 smithi146 ceph-mon[33158]: pgmap v416: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:08.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:07:08.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[29849]: osdmap e299: 8 total, 8 up, 8 in 2024-09-11T08:07:08.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:08.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2989381855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:08.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[29849]: from='client.46988 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:08.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-11T08:07:08.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-11T08:07:08.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[29849]: from='client.46988 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi102-80356-58"}]': finished 2024-09-11T08:07:08.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-11T08:07:08.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[29849]: osdmap e300: 8 total, 8 up, 8 in 2024-09-11T08:07:08.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2989381855' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:08.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[29849]: from='client.46988 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:08.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:07:08.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[41371]: osdmap e299: 8 total, 8 up, 8 in 2024-09-11T08:07:08.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:08.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2989381855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:08.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[41371]: from='client.46988 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:08.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-11T08:07:08.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-11T08:07:08.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[41371]: from='client.46988 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi102-80356-58"}]': finished 2024-09-11T08:07:08.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-11T08:07:08.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[41371]: osdmap e300: 8 total, 8 up, 8 in 2024-09-11T08:07:08.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2989381855' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:08.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:08 smithi102 ceph-mon[41371]: from='client.46988 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:08.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:08 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:07:08.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:08 smithi146 ceph-mon[33158]: osdmap e299: 8 total, 8 up, 8 in 2024-09-11T08:07:08.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:08 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:08.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:08 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2989381855' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:08.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:08 smithi146 ceph-mon[33158]: from='client.46988 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:08.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:08 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-11T08:07:08.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:08 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-11T08:07:08.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:08 smithi146 ceph-mon[33158]: from='client.46988 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi102-80356-58"}]': finished 2024-09-11T08:07:08.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:08 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-11T08:07:08.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:08 smithi146 ceph-mon[33158]: osdmap e300: 8 total, 8 up, 8 in 2024-09-11T08:07:08.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:08 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2989381855' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:08.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:08 smithi146 ceph-mon[33158]: from='client.46988 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi102-80356-58"}]: dispatch 2024-09-11T08:07:09.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:09 smithi102 ceph-mon[29849]: pgmap v419: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:09.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:09 smithi102 ceph-mon[29849]: from='client.46988 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi102-80356-58"}]': finished 2024-09-11T08:07:09.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:09 smithi102 ceph-mon[29849]: osdmap e301: 8 total, 8 up, 8 in 2024-09-11T08:07:09.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:09 smithi102 ceph-mon[41371]: pgmap v419: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:09.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:09 smithi102 ceph-mon[41371]: from='client.46988 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi102-80356-58"}]': finished 2024-09-11T08:07:09.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:09 smithi102 ceph-mon[41371]: osdmap e301: 8 total, 8 up, 8 in 2024-09-11T08:07:09.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:09 smithi146 ceph-mon[33158]: pgmap v419: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:09.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:09 smithi146 ceph-mon[33158]: from='client.46988 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi102-80356-58"}]': finished 2024-09-11T08:07:09.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:09 smithi146 ceph-mon[33158]: osdmap e301: 8 total, 8 up, 8 in 2024-09-11T08:07:09.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:07:09 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:07:10.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:07:09 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:07:09] "GET /metrics HTTP/1.1" 200 230924 "" "Prometheus/2.43.0" 2024-09-11T08:07:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi102-80356-59"}]: dispatch 2024-09-11T08:07:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi102-80356-59"}]: dispatch 2024-09-11T08:07:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi102-80356-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:10 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi102-80356-59"}]: dispatch 2024-09-11T08:07:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi102-80356-59"}]: dispatch 2024-09-11T08:07:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi102-80356-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:10.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:10 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi102-80356-59"}]: dispatch 2024-09-11T08:07:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi102-80356-59"}]: dispatch 2024-09-11T08:07:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi102-80356-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:10 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:11.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:11 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi102-80356-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:11.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:11 smithi102 ceph-mon[29849]: osdmap e302: 8 total, 8 up, 8 in 2024-09-11T08:07:11.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:11 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi102-80356-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi102-80356-59"}]: dispatch 2024-09-11T08:07:11.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:11 smithi102 ceph-mon[29849]: pgmap v422: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-11T08:07:11.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:11 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi102-80356-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:11.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:11 smithi102 ceph-mon[41371]: osdmap e302: 8 total, 8 up, 8 in 2024-09-11T08:07:11.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:11 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi102-80356-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi102-80356-59"}]: dispatch 2024-09-11T08:07:11.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:11 smithi102 ceph-mon[41371]: pgmap v422: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-11T08:07:11.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:11 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi102-80356-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:11.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:11 smithi146 ceph-mon[33158]: osdmap e302: 8 total, 8 up, 8 in 2024-09-11T08:07:11.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:11 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi102-80356-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi102-80356-59"}]: dispatch 2024-09-11T08:07:11.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:11 smithi146 ceph-mon[33158]: pgmap v422: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-11T08:07:12.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:12 smithi102 ceph-mon[29849]: osdmap e303: 8 total, 8 up, 8 in 2024-09-11T08:07:12.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi102-80356-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi102-80356-59"}]': finished 2024-09-11T08:07:12.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:12 smithi102 ceph-mon[29849]: osdmap e304: 8 total, 8 up, 8 in 2024-09-11T08:07:12.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:12 smithi102 ceph-mon[41371]: osdmap e303: 8 total, 8 up, 8 in 2024-09-11T08:07:12.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi102-80356-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi102-80356-59"}]': finished 2024-09-11T08:07:12.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:12 smithi102 ceph-mon[41371]: osdmap e304: 8 total, 8 up, 8 in 2024-09-11T08:07:12.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:12 smithi146 ceph-mon[33158]: osdmap e303: 8 total, 8 up, 8 in 2024-09-11T08:07:12.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi102-80356-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi102-80356-59"}]': finished 2024-09-11T08:07:12.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:12 smithi146 ceph-mon[33158]: osdmap e304: 8 total, 8 up, 8 in 2024-09-11T08:07:13.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:13 smithi102 ceph-mon[29849]: pgmap v425: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-11T08:07:13.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:13 smithi102 ceph-mon[29849]: osdmap e305: 8 total, 8 up, 8 in 2024-09-11T08:07:13.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:13 smithi102 ceph-mon[41371]: pgmap v425: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-11T08:07:13.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:13 smithi102 ceph-mon[41371]: osdmap e305: 8 total, 8 up, 8 in 2024-09-11T08:07:13.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:13 smithi146 ceph-mon[33158]: pgmap v425: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-11T08:07:13.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:13 smithi146 ceph-mon[33158]: osdmap e305: 8 total, 8 up, 8 in 2024-09-11T08:07:14.588 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:14 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:14.589 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:14 smithi146 ceph-mon[33158]: osdmap e306: 8 total, 8 up, 8 in 2024-09-11T08:07:14.589 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi102-80356-59"}]: dispatch 2024-09-11T08:07:14.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:14 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:14.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:14 smithi102 ceph-mon[29849]: osdmap e306: 8 total, 8 up, 8 in 2024-09-11T08:07:14.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi102-80356-59"}]: dispatch 2024-09-11T08:07:14.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:14 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:14.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:14 smithi102 ceph-mon[41371]: osdmap e306: 8 total, 8 up, 8 in 2024-09-11T08:07:14.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi102-80356-59"}]: dispatch 2024-09-11T08:07:14.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:07:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:07:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:07:15.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:07:15.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:15 smithi102 ceph-mon[29849]: pgmap v428: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:15.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:15 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi102-80356-59"}]': finished 2024-09-11T08:07:15.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:15 smithi102 ceph-mon[29849]: osdmap e307: 8 total, 8 up, 8 in 2024-09-11T08:07:15.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:15 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi102-80356-59"}]: dispatch 2024-09-11T08:07:15.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:07:15.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:15 smithi102 ceph-mon[41371]: pgmap v428: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:15.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:15 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi102-80356-59"}]': finished 2024-09-11T08:07:15.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:15 smithi102 ceph-mon[41371]: osdmap e307: 8 total, 8 up, 8 in 2024-09-11T08:07:15.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:15 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi102-80356-59"}]: dispatch 2024-09-11T08:07:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:07:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:15 smithi146 ceph-mon[33158]: pgmap v428: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:15 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi102-80356-59"}]': finished 2024-09-11T08:07:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:15 smithi146 ceph-mon[33158]: osdmap e307: 8 total, 8 up, 8 in 2024-09-11T08:07:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:15 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi102-80356-59"}]: dispatch 2024-09-11T08:07:16.032 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:7462ddf6:::1.RoundTripAppendPP (2966 ms) 2024-09-11T08:07:16.032 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-09-11T08:07:16.032 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3015 ms) 2024-09-11T08:07:16.033 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-09-11T08:07:16.033 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3016 ms) 2024-09-11T08:07:16.033 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-09-11T08:07:16.033 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3009 ms) 2024-09-11T08:07:16.033 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-09-11T08:07:16.034 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: setting pool EIO 2024-09-11T08:07:16.034 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: max_success 1657, min_failed 1658 2024-09-11T08:07:16.034 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (4030 ms) 2024-09-11T08:07:16.034 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-09-11T08:07:16.034 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3021 ms) 2024-09-11T08:07:16.035 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (120844 ms total) 2024-09-11T08:07:16.035 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: 2024-09-11T08:07:16.035 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-09-11T08:07:16.035 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-09-11T08:07:16.035 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3259 ms) 2024-09-11T08:07:16.035 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-09-11T08:07:16.036 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (8798 ms) 2024-09-11T08:07:16.036 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-09-11T08:07:16.036 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14863 ms) 2024-09-11T08:07:16.036 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-09-11T08:07:16.036 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3029 ms) 2024-09-11T08:07:16.037 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (29949 ms total) 2024-09-11T08:07:16.037 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: 2024-09-11T08:07:16.037 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-09-11T08:07:16.037 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-09-11T08:07:16.037 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3004 ms) 2024-09-11T08:07:16.038 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3004 ms total) 2024-09-11T08:07:16.038 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: 2024-09-11T08:07:16.038 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-09-11T08:07:16.038 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-09-11T08:07:16.038 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (13977 ms) 2024-09-11T08:07:16.038 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-09-11T08:07:16.039 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (6991 ms) 2024-09-11T08:07:16.039 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-09-11T08:07:16.039 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7042 ms) 2024-09-11T08:07:16.039 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-09-11T08:07:16.039 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7035 ms) 2024-09-11T08:07:16.040 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-09-11T08:07:16.040 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3010 ms) 2024-09-11T08:07:16.040 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-09-11T08:07:16.040 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7022 ms) 2024-09-11T08:07:16.040 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-09-11T08:07:16.041 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (6897 ms) 2024-09-11T08:07:16.041 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-09-11T08:07:16.041 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7025 ms) 2024-09-11T08:07:16.041 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-09-11T08:07:16.041 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7019 ms) 2024-09-11T08:07:16.042 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-09-11T08:07:16.042 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7020 ms) 2024-09-11T08:07:16.042 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-09-11T08:07:16.042 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7016 ms) 2024-09-11T08:07:16.042 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-09-11T08:07:16.043 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7011 ms) 2024-09-11T08:07:16.043 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-09-11T08:07:16.043 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7032 ms) 2024-09-11T08:07:17.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi102-80356-59"}]': finished 2024-09-11T08:07:17.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:17 smithi102 ceph-mon[29849]: osdmap e308: 8 total, 8 up, 8 in 2024-09-11T08:07:17.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:17 smithi102 ceph-mon[29849]: pgmap v431: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-11T08:07:17.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi102-80356-59"}]': finished 2024-09-11T08:07:17.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:17 smithi102 ceph-mon[41371]: osdmap e308: 8 total, 8 up, 8 in 2024-09-11T08:07:17.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:17 smithi102 ceph-mon[41371]: pgmap v431: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-11T08:07:17.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2280720115' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi102-80356-59"}]': finished 2024-09-11T08:07:17.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:17 smithi146 ceph-mon[33158]: osdmap e308: 8 total, 8 up, 8 in 2024-09-11T08:07:17.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:17 smithi146 ceph-mon[33158]: pgmap v431: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-11T08:07:18.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:18 smithi102 ceph-mon[29849]: osdmap e309: 8 total, 8 up, 8 in 2024-09-11T08:07:18.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/459470456' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi102-80356-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:18.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:18 smithi102 ceph-mon[41371]: osdmap e309: 8 total, 8 up, 8 in 2024-09-11T08:07:18.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/459470456' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi102-80356-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:18.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:18 smithi146 ceph-mon[33158]: osdmap e309: 8 total, 8 up, 8 in 2024-09-11T08:07:18.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/459470456' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi102-80356-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:19.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/459470456' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi102-80356-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:07:19.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:19 smithi102 ceph-mon[29849]: osdmap e310: 8 total, 8 up, 8 in 2024-09-11T08:07:19.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:19 smithi102 ceph-mon[29849]: pgmap v434: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-11T08:07:19.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/459470456' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi102-80356-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:07:19.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:19 smithi102 ceph-mon[41371]: osdmap e310: 8 total, 8 up, 8 in 2024-09-11T08:07:19.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:19 smithi102 ceph-mon[41371]: pgmap v434: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-11T08:07:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/459470456' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi102-80356-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:07:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:19 smithi146 ceph-mon[33158]: osdmap e310: 8 total, 8 up, 8 in 2024-09-11T08:07:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:19 smithi146 ceph-mon[33158]: pgmap v434: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-11T08:07:19.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:07:19 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:07:20.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:07:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:07:19] "GET /metrics HTTP/1.1" 200 228532 "" "Prometheus/2.43.0" 2024-09-11T08:07:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[29849]: osdmap e311: 8 total, 8 up, 8 in 2024-09-11T08:07:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1111733754' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[29849]: from='client.47006 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1111733754' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[29849]: from='client.47006 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1111733754' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi102-80356-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[29849]: from='client.47006 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi102-80356-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:20.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:20.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:20.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:20.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-45"}]: dispatch 2024-09-11T08:07:20.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-45"}]: dispatch 2024-09-11T08:07:20.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[41371]: osdmap e311: 8 total, 8 up, 8 in 2024-09-11T08:07:20.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1111733754' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:20.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[41371]: from='client.47006 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:20.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1111733754' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:20.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[41371]: from='client.47006 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:20.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1111733754' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi102-80356-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:20.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[41371]: from='client.47006 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi102-80356-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:20.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:20.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:20.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:20.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-45"}]: dispatch 2024-09-11T08:07:20.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:20 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-45"}]: dispatch 2024-09-11T08:07:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:20 smithi146 ceph-mon[33158]: osdmap e311: 8 total, 8 up, 8 in 2024-09-11T08:07:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1111733754' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:20 smithi146 ceph-mon[33158]: from='client.47006 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1111733754' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:20 smithi146 ceph-mon[33158]: from='client.47006 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1111733754' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi102-80356-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:20 smithi146 ceph-mon[33158]: from='client.47006 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi102-80356-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:20 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:20 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:20.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-45"}]: dispatch 2024-09-11T08:07:20.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:20 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-45"}]: dispatch 2024-09-11T08:07:21.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:21 smithi102 ceph-mon[29849]: from='client.47006 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi102-80356-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:21 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-45"}]': finished 2024-09-11T08:07:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:21 smithi102 ceph-mon[29849]: osdmap e312: 8 total, 8 up, 8 in 2024-09-11T08:07:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:21 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1111733754' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi102-80356-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:21 smithi102 ceph-mon[29849]: from='client.47006 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi102-80356-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:21 smithi102 ceph-mon[29849]: pgmap v437: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:07:21.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:21 smithi102 ceph-mon[41371]: from='client.47006 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi102-80356-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:21.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:21 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-45"}]': finished 2024-09-11T08:07:21.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:21 smithi102 ceph-mon[41371]: osdmap e312: 8 total, 8 up, 8 in 2024-09-11T08:07:21.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:21 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1111733754' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi102-80356-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:21.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:21 smithi102 ceph-mon[41371]: from='client.47006 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi102-80356-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:21.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:21 smithi102 ceph-mon[41371]: pgmap v437: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:07:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:21 smithi146 ceph-mon[33158]: from='client.47006 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi102-80356-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:21 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-45"}]': finished 2024-09-11T08:07:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:21 smithi146 ceph-mon[33158]: osdmap e312: 8 total, 8 up, 8 in 2024-09-11T08:07:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:21 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1111733754' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi102-80356-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:21 smithi146 ceph-mon[33158]: from='client.47006 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi102-80356-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:21 smithi146 ceph-mon[33158]: pgmap v437: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:07:22.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:22 smithi102 ceph-mon[29849]: osdmap e313: 8 total, 8 up, 8 in 2024-09-11T08:07:22.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:22 smithi102 ceph-mon[29849]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:22 smithi102 ceph-mon[41371]: osdmap e313: 8 total, 8 up, 8 in 2024-09-11T08:07:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:22 smithi102 ceph-mon[41371]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:22.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:22 smithi146 ceph-mon[33158]: osdmap e313: 8 total, 8 up, 8 in 2024-09-11T08:07:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:22 smithi146 ceph-mon[33158]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:23.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:23 smithi102 ceph-mon[41371]: from='client.47006 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi102-80356-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi102-80356-61"}]': finished 2024-09-11T08:07:23.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:23 smithi102 ceph-mon[41371]: osdmap e314: 8 total, 8 up, 8 in 2024-09-11T08:07:23.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:23.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:23 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:23.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:23 smithi102 ceph-mon[41371]: pgmap v440: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:23.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:23 smithi102 ceph-mon[29849]: from='client.47006 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi102-80356-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi102-80356-61"}]': finished 2024-09-11T08:07:23.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:23 smithi102 ceph-mon[29849]: osdmap e314: 8 total, 8 up, 8 in 2024-09-11T08:07:23.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:23.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:23 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:23.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:23 smithi102 ceph-mon[29849]: pgmap v440: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:23.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:23 smithi146 ceph-mon[33158]: from='client.47006 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi102-80356-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi102-80356-61"}]': finished 2024-09-11T08:07:23.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:23 smithi146 ceph-mon[33158]: osdmap e314: 8 total, 8 up, 8 in 2024-09-11T08:07:23.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:23.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:23 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:23.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:23 smithi146 ceph-mon[33158]: pgmap v440: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:24 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:07:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:24 smithi102 ceph-mon[41371]: osdmap e315: 8 total, 8 up, 8 in 2024-09-11T08:07:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:24 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:24.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:24 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:07:24.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:24 smithi102 ceph-mon[29849]: osdmap e315: 8 total, 8 up, 8 in 2024-09-11T08:07:24.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:24.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:24 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:24 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:07:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:24 smithi146 ceph-mon[33158]: osdmap e315: 8 total, 8 up, 8 in 2024-09-11T08:07:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:24 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:24.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:07:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:07:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:07:25.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:25 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:07:25.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:25 smithi102 ceph-mon[41371]: osdmap e316: 8 total, 8 up, 8 in 2024-09-11T08:07:25.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-47"}]: dispatch 2024-09-11T08:07:25.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1111733754' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:25.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:25 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-47"}]: dispatch 2024-09-11T08:07:25.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:25 smithi102 ceph-mon[41371]: from='client.47006 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:25.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:25 smithi102 ceph-mon[41371]: pgmap v443: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:25 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:07:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:25 smithi102 ceph-mon[29849]: osdmap e316: 8 total, 8 up, 8 in 2024-09-11T08:07:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-47"}]: dispatch 2024-09-11T08:07:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1111733754' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:25 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-47"}]: dispatch 2024-09-11T08:07:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:25 smithi102 ceph-mon[29849]: from='client.47006 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:25.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:25 smithi102 ceph-mon[29849]: pgmap v443: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:25 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:07:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:25 smithi146 ceph-mon[33158]: osdmap e316: 8 total, 8 up, 8 in 2024-09-11T08:07:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-47"}]: dispatch 2024-09-11T08:07:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1111733754' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:25 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-47"}]: dispatch 2024-09-11T08:07:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:25 smithi146 ceph-mon[33158]: from='client.47006 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:25 smithi146 ceph-mon[33158]: pgmap v443: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:26.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:26 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-47"}]': finished 2024-09-11T08:07:26.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:26 smithi102 ceph-mon[41371]: from='client.47006 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi102-80356-61"}]': finished 2024-09-11T08:07:26.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:26 smithi102 ceph-mon[41371]: osdmap e317: 8 total, 8 up, 8 in 2024-09-11T08:07:26.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-47", "mode": "writeback"}]: dispatch 2024-09-11T08:07:26.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:26 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-47", "mode": "writeback"}]: dispatch 2024-09-11T08:07:26.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1111733754' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:26.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:26 smithi102 ceph-mon[41371]: from='client.47006 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:26.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:26 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-47"}]': finished 2024-09-11T08:07:26.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:26 smithi102 ceph-mon[29849]: from='client.47006 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi102-80356-61"}]': finished 2024-09-11T08:07:26.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:26 smithi102 ceph-mon[29849]: osdmap e317: 8 total, 8 up, 8 in 2024-09-11T08:07:26.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-47", "mode": "writeback"}]: dispatch 2024-09-11T08:07:26.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:26 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-47", "mode": "writeback"}]: dispatch 2024-09-11T08:07:26.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1111733754' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:26.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:26 smithi102 ceph-mon[29849]: from='client.47006 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:26.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:26 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-47"}]': finished 2024-09-11T08:07:26.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:26 smithi146 ceph-mon[33158]: from='client.47006 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi102-80356-61"}]': finished 2024-09-11T08:07:26.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:26 smithi146 ceph-mon[33158]: osdmap e317: 8 total, 8 up, 8 in 2024-09-11T08:07:26.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-47", "mode": "writeback"}]: dispatch 2024-09-11T08:07:26.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:26 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-47", "mode": "writeback"}]: dispatch 2024-09-11T08:07:26.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1111733754' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:26.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:26 smithi146 ceph-mon[33158]: from='client.47006 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi102-80356-61"}]: dispatch 2024-09-11T08:07:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:27 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:07:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:27 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-47", "mode": "writeback"}]': finished 2024-09-11T08:07:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:27 smithi102 ceph-mon[29849]: from='client.47006 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi102-80356-61"}]': finished 2024-09-11T08:07:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:27 smithi102 ceph-mon[29849]: osdmap e318: 8 total, 8 up, 8 in 2024-09-11T08:07:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:07:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:27 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:07:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi102-80356-62"}]: dispatch 2024-09-11T08:07:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi102-80356-62"}]: dispatch 2024-09-11T08:07:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi102-80356-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:27 smithi102 ceph-mon[29849]: pgmap v446: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-11T08:07:27.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:27 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:07:27.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:27 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-47", "mode": "writeback"}]': finished 2024-09-11T08:07:27.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:27 smithi102 ceph-mon[41371]: from='client.47006 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi102-80356-61"}]': finished 2024-09-11T08:07:27.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:27 smithi102 ceph-mon[41371]: osdmap e318: 8 total, 8 up, 8 in 2024-09-11T08:07:27.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:07:27.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:27 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:07:27.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi102-80356-62"}]: dispatch 2024-09-11T08:07:27.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi102-80356-62"}]: dispatch 2024-09-11T08:07:27.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi102-80356-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:27.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:27 smithi102 ceph-mon[41371]: pgmap v446: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-11T08:07:27.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:27 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:07:27.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:27 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-47", "mode": "writeback"}]': finished 2024-09-11T08:07:27.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:27 smithi146 ceph-mon[33158]: from='client.47006 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi102-80356-61"}]': finished 2024-09-11T08:07:27.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:27 smithi146 ceph-mon[33158]: osdmap e318: 8 total, 8 up, 8 in 2024-09-11T08:07:27.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:07:27.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:27 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:07:27.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi102-80356-62"}]: dispatch 2024-09-11T08:07:27.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi102-80356-62"}]: dispatch 2024-09-11T08:07:27.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi102-80356-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:27.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:27 smithi146 ceph-mon[33158]: pgmap v446: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-11T08:07:28.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:28 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:07:28.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi102-80356-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:28.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:28 smithi102 ceph-mon[41371]: osdmap e319: 8 total, 8 up, 8 in 2024-09-11T08:07:28.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:07:28.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi102-80356-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi102-80356-62"}]: dispatch 2024-09-11T08:07:28.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:28 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:07:28.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:28 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:07:28.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi102-80356-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:28.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:28 smithi102 ceph-mon[29849]: osdmap e319: 8 total, 8 up, 8 in 2024-09-11T08:07:28.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:07:28.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi102-80356-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi102-80356-62"}]: dispatch 2024-09-11T08:07:28.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:28 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:07:28.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:28 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:07:28.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi102-80356-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:28.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:28 smithi146 ceph-mon[33158]: osdmap e319: 8 total, 8 up, 8 in 2024-09-11T08:07:28.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:07:28.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi102-80356-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi102-80356-62"}]: dispatch 2024-09-11T08:07:28.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:28 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:07:29.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:29 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:07:29.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:29 smithi102 ceph-mon[41371]: osdmap e320: 8 total, 8 up, 8 in 2024-09-11T08:07:29.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:07:29.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:29 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:07:29.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:29 smithi102 ceph-mon[41371]: pgmap v449: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-11T08:07:29.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:29 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:07:29.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi102-80356-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi102-80356-62"}]': finished 2024-09-11T08:07:29.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:29 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:07:29.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:29 smithi102 ceph-mon[41371]: osdmap e321: 8 total, 8 up, 8 in 2024-09-11T08:07:29.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:07:29.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:29 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:07:29.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:29 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:07:29.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:29 smithi102 ceph-mon[29849]: osdmap e320: 8 total, 8 up, 8 in 2024-09-11T08:07:29.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:07:29.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:29 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:07:29.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:29 smithi102 ceph-mon[29849]: pgmap v449: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-11T08:07:29.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:29 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:07:29.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi102-80356-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi102-80356-62"}]': finished 2024-09-11T08:07:29.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:29 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:07:29.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:29 smithi102 ceph-mon[29849]: osdmap e321: 8 total, 8 up, 8 in 2024-09-11T08:07:29.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:07:29.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:29 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:07:29.664 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:29 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:07:29.664 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:29 smithi146 ceph-mon[33158]: osdmap e320: 8 total, 8 up, 8 in 2024-09-11T08:07:29.664 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:07:29.664 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:29 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:07:29.665 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:29 smithi146 ceph-mon[33158]: pgmap v449: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-11T08:07:29.665 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:29 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:07:29.665 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi102-80356-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi102-80356-62"}]': finished 2024-09-11T08:07:29.665 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:29 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:07:29.665 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:29 smithi146 ceph-mon[33158]: osdmap e321: 8 total, 8 up, 8 in 2024-09-11T08:07:29.665 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:07:29.665 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:29 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:07:29.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:07:29 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:07:30.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:07:29 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:07:29] "GET /metrics HTTP/1.1" 200 230941 "" "Prometheus/2.43.0" 2024-09-11T08:07:30.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:30 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:07:30.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:30 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:30.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:30 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:07:30.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:30 smithi102 ceph-mon[41371]: osdmap e322: 8 total, 8 up, 8 in 2024-09-11T08:07:30.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:30 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-11T08:07:30.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:30 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-11T08:07:30.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:30 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:07:30.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:30 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:30.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:30 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:07:30.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:30 smithi102 ceph-mon[29849]: osdmap e322: 8 total, 8 up, 8 in 2024-09-11T08:07:30.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:30 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-11T08:07:30.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:30 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-11T08:07:30.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:30 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:07:30.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:30 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:30.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:30 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:07:30.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:30 smithi146 ceph-mon[33158]: osdmap e322: 8 total, 8 up, 8 in 2024-09-11T08:07:30.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:30 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-11T08:07:30.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:30 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-11T08:07:31.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:31 smithi102 ceph-mon[41371]: pgmap v452: 236 pgs: 236 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:07:31.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:31 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-11T08:07:31.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:31 smithi102 ceph-mon[41371]: osdmap e323: 8 total, 8 up, 8 in 2024-09-11T08:07:31.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:31 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-11T08:07:31.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:31 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-11T08:07:31.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:31 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi102-80356-62"}]: dispatch 2024-09-11T08:07:31.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:31 smithi102 ceph-mon[29849]: pgmap v452: 236 pgs: 236 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:07:31.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:31 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-11T08:07:31.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:31 smithi102 ceph-mon[29849]: osdmap e323: 8 total, 8 up, 8 in 2024-09-11T08:07:31.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:31 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-11T08:07:31.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:31 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-11T08:07:31.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:31 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi102-80356-62"}]: dispatch 2024-09-11T08:07:31.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:31 smithi146 ceph-mon[33158]: pgmap v452: 236 pgs: 236 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:07:31.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:31 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-11T08:07:31.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:31 smithi146 ceph-mon[33158]: osdmap e323: 8 total, 8 up, 8 in 2024-09-11T08:07:31.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:31 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-11T08:07:31.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:31 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-11T08:07:31.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:31 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi102-80356-62"}]: dispatch 2024-09-11T08:07:32.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:32 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:32.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:32 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-11T08:07:32.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi102-80356-62"}]': finished 2024-09-11T08:07:32.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:32 smithi102 ceph-mon[41371]: osdmap e324: 8 total, 8 up, 8 in 2024-09-11T08:07:32.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi102-80356-62"}]: dispatch 2024-09-11T08:07:32.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:32.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:32 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:32.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:32 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:32.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:32 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-11T08:07:32.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi102-80356-62"}]': finished 2024-09-11T08:07:32.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:32 smithi102 ceph-mon[29849]: osdmap e324: 8 total, 8 up, 8 in 2024-09-11T08:07:32.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi102-80356-62"}]: dispatch 2024-09-11T08:07:32.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:32.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:32 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:32.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:32 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:32.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:32 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-11T08:07:32.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi102-80356-62"}]': finished 2024-09-11T08:07:32.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:32 smithi146 ceph-mon[33158]: osdmap e324: 8 total, 8 up, 8 in 2024-09-11T08:07:32.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi102-80356-62"}]: dispatch 2024-09-11T08:07:32.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:32.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:32 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:33.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[41371]: pgmap v455: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:33.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi102-80356-62"}]': finished 2024-09-11T08:07:33.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:07:33.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[41371]: osdmap e325: 8 total, 8 up, 8 in 2024-09-11T08:07:33.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47"}]: dispatch 2024-09-11T08:07:33.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47"}]: dispatch 2024-09-11T08:07:33.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3190136703' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:33.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[41371]: from='client.47018 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:33.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3190136703' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:33.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[41371]: from='client.47018 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:33.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3190136703' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi102-80356-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:33.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[41371]: from='client.47018 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi102-80356-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:33.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[29849]: pgmap v455: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:33.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi102-80356-62"}]': finished 2024-09-11T08:07:33.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:07:33.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[29849]: osdmap e325: 8 total, 8 up, 8 in 2024-09-11T08:07:33.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47"}]: dispatch 2024-09-11T08:07:33.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47"}]: dispatch 2024-09-11T08:07:33.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3190136703' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:33.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[29849]: from='client.47018 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:33.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3190136703' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:33.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[29849]: from='client.47018 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:33.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3190136703' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi102-80356-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:33.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:33 smithi102 ceph-mon[29849]: from='client.47018 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi102-80356-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:33.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:33 smithi146 ceph-mon[33158]: pgmap v455: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:33.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4090796504' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi102-80356-62"}]': finished 2024-09-11T08:07:33.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:33 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:07:33.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:33 smithi146 ceph-mon[33158]: osdmap e325: 8 total, 8 up, 8 in 2024-09-11T08:07:33.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47"}]: dispatch 2024-09-11T08:07:33.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:33 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47"}]: dispatch 2024-09-11T08:07:33.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3190136703' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:33.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:33 smithi146 ceph-mon[33158]: from='client.47018 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:33.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3190136703' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:33.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:33 smithi146 ceph-mon[33158]: from='client.47018 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:33.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3190136703' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi102-80356-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:33.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:33 smithi146 ceph-mon[33158]: from='client.47018 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi102-80356-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:34.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:07:34 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:07:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:07:35.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:35 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47"}]': finished 2024-09-11T08:07:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:35 smithi102 ceph-mon[29849]: from='client.47018 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi102-80356-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:35 smithi102 ceph-mon[29849]: osdmap e326: 8 total, 8 up, 8 in 2024-09-11T08:07:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3190136703' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi102-80356-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:35 smithi102 ceph-mon[29849]: from='client.47018 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi102-80356-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:35 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:35.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47"}]: dispatch 2024-09-11T08:07:35.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:35 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47"}]: dispatch 2024-09-11T08:07:35.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:35 smithi102 ceph-mon[29849]: pgmap v458: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:35.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:35 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47"}]': finished 2024-09-11T08:07:35.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:35 smithi102 ceph-mon[41371]: from='client.47018 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi102-80356-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:35.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:35 smithi102 ceph-mon[41371]: osdmap e326: 8 total, 8 up, 8 in 2024-09-11T08:07:35.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3190136703' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi102-80356-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:35.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:35 smithi102 ceph-mon[41371]: from='client.47018 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi102-80356-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:35.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:35.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:35 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:35.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47"}]: dispatch 2024-09-11T08:07:35.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:35 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47"}]: dispatch 2024-09-11T08:07:35.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:35 smithi102 ceph-mon[41371]: pgmap v458: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:35.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:35 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47"}]': finished 2024-09-11T08:07:35.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:35 smithi146 ceph-mon[33158]: from='client.47018 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi102-80356-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:35.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:35 smithi146 ceph-mon[33158]: osdmap e326: 8 total, 8 up, 8 in 2024-09-11T08:07:35.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3190136703' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi102-80356-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:35.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:35 smithi146 ceph-mon[33158]: from='client.47018 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi102-80356-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:35.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:35.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:35 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:35.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47"}]: dispatch 2024-09-11T08:07:35.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:35 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-47"}]: dispatch 2024-09-11T08:07:35.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:35 smithi146 ceph-mon[33158]: pgmap v458: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:36.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:36 smithi102 ceph-mon[29849]: osdmap e327: 8 total, 8 up, 8 in 2024-09-11T08:07:36.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:36 smithi102 ceph-mon[41371]: osdmap e327: 8 total, 8 up, 8 in 2024-09-11T08:07:36.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:36 smithi146 ceph-mon[33158]: osdmap e327: 8 total, 8 up, 8 in 2024-09-11T08:07:37.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:37 smithi102 ceph-mon[41371]: from='client.47018 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi102-80356-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi102-80356-63"}]': finished 2024-09-11T08:07:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:37 smithi102 ceph-mon[41371]: osdmap e328: 8 total, 8 up, 8 in 2024-09-11T08:07:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:37 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:37 smithi102 ceph-mon[41371]: pgmap v461: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:07:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:37 smithi102 ceph-mon[41371]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:37 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:07:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:37 smithi102 ceph-mon[41371]: osdmap e329: 8 total, 8 up, 8 in 2024-09-11T08:07:37.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:37 smithi102 ceph-mon[29849]: from='client.47018 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi102-80356-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi102-80356-63"}]': finished 2024-09-11T08:07:37.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:37 smithi102 ceph-mon[29849]: osdmap e328: 8 total, 8 up, 8 in 2024-09-11T08:07:37.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:37.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:37 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:37.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:37 smithi102 ceph-mon[29849]: pgmap v461: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:07:37.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:37 smithi102 ceph-mon[29849]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:37.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:37 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:07:37.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:37 smithi102 ceph-mon[29849]: osdmap e329: 8 total, 8 up, 8 in 2024-09-11T08:07:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:37 smithi146 ceph-mon[33158]: from='client.47018 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi102-80356-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi102-80356-63"}]': finished 2024-09-11T08:07:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:37 smithi146 ceph-mon[33158]: osdmap e328: 8 total, 8 up, 8 in 2024-09-11T08:07:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:37 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:37 smithi146 ceph-mon[33158]: pgmap v461: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:07:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:37 smithi146 ceph-mon[33158]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:37 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:07:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:37 smithi146 ceph-mon[33158]: osdmap e329: 8 total, 8 up, 8 in 2024-09-11T08:07:38.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:38.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:38 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:38.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:38 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:07:38.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:38.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:38 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:38.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:38 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:07:38.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:38.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:38 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:38.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:38 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:07:39.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:39 smithi146 ceph-mon[33158]: osdmap e330: 8 total, 8 up, 8 in 2024-09-11T08:07:39.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:39 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-49"}]: dispatch 2024-09-11T08:07:39.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:39 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3190136703' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:39.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:39 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-49"}]: dispatch 2024-09-11T08:07:39.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:39 smithi146 ceph-mon[33158]: from='client.47018 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:39.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:39 smithi146 ceph-mon[33158]: pgmap v464: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:07:39.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:39 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-49"}]': finished 2024-09-11T08:07:39.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:39 smithi146 ceph-mon[33158]: from='client.47018 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi102-80356-63"}]': finished 2024-09-11T08:07:39.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:39 smithi146 ceph-mon[33158]: osdmap e331: 8 total, 8 up, 8 in 2024-09-11T08:07:39.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:39 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-49", "mode": "readproxy"}]: dispatch 2024-09-11T08:07:39.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:39 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-49", "mode": "readproxy"}]: dispatch 2024-09-11T08:07:39.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:39 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3190136703' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:39.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[29849]: osdmap e330: 8 total, 8 up, 8 in 2024-09-11T08:07:39.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-49"}]: dispatch 2024-09-11T08:07:39.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3190136703' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:39.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-49"}]: dispatch 2024-09-11T08:07:39.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[29849]: from='client.47018 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:39.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[29849]: pgmap v464: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:07:39.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-49"}]': finished 2024-09-11T08:07:39.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[29849]: from='client.47018 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi102-80356-63"}]': finished 2024-09-11T08:07:39.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[29849]: osdmap e331: 8 total, 8 up, 8 in 2024-09-11T08:07:39.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-49", "mode": "readproxy"}]: dispatch 2024-09-11T08:07:39.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-49", "mode": "readproxy"}]: dispatch 2024-09-11T08:07:39.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3190136703' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:39.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[41371]: osdmap e330: 8 total, 8 up, 8 in 2024-09-11T08:07:39.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-49"}]: dispatch 2024-09-11T08:07:39.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3190136703' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:39.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-49"}]: dispatch 2024-09-11T08:07:39.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[41371]: from='client.47018 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:39.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[41371]: pgmap v464: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:07:39.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-49"}]': finished 2024-09-11T08:07:39.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[41371]: from='client.47018 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi102-80356-63"}]': finished 2024-09-11T08:07:39.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[41371]: osdmap e331: 8 total, 8 up, 8 in 2024-09-11T08:07:39.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-49", "mode": "readproxy"}]: dispatch 2024-09-11T08:07:39.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-49", "mode": "readproxy"}]: dispatch 2024-09-11T08:07:39.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3190136703' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:39.935 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:07:39 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:07:40.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:07:39 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:07:39] "GET /metrics HTTP/1.1" 200 230941 "" "Prometheus/2.43.0" 2024-09-11T08:07:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:40 smithi146 ceph-mon[33158]: from='client.47018 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:40 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:40 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:07:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:40 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:07:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:40 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-49", "mode": "readproxy"}]': finished 2024-09-11T08:07:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:40 smithi146 ceph-mon[33158]: from='client.47018 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi102-80356-63"}]': finished 2024-09-11T08:07:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:40 smithi146 ceph-mon[33158]: osdmap e332: 8 total, 8 up, 8 in 2024-09-11T08:07:40.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:40 smithi102 ceph-mon[41371]: from='client.47018 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:40.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:40 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:40.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:40 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:07:40.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:40 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:07:40.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:40 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-49", "mode": "readproxy"}]': finished 2024-09-11T08:07:40.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:40 smithi102 ceph-mon[41371]: from='client.47018 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi102-80356-63"}]': finished 2024-09-11T08:07:40.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:40 smithi102 ceph-mon[41371]: osdmap e332: 8 total, 8 up, 8 in 2024-09-11T08:07:40.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:40 smithi102 ceph-mon[29849]: from='client.47018 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi102-80356-63"}]: dispatch 2024-09-11T08:07:40.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:40 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:40.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:40 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:07:40.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:40 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:07:40.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:40 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-49", "mode": "readproxy"}]': finished 2024-09-11T08:07:40.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:40 smithi102 ceph-mon[29849]: from='client.47018 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi102-80356-63"}]': finished 2024-09-11T08:07:40.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:40 smithi102 ceph-mon[29849]: osdmap e332: 8 total, 8 up, 8 in 2024-09-11T08:07:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/763872328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:41 smithi146 ceph-mon[33158]: from='client.48124 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/763872328' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:41 smithi146 ceph-mon[33158]: from='client.48124 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/763872328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi102-80356-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:41 smithi146 ceph-mon[33158]: from='client.48124 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi102-80356-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:41 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:07:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:41 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:07:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:41 smithi146 ceph-mon[33158]: pgmap v467: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:07:41.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:41 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:41.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:41 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:07:41.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:41 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:07:41.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/763872328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:41.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[41371]: from='client.48124 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:41.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/763872328' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:41.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[41371]: from='client.48124 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:41.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/763872328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi102-80356-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:41.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[41371]: from='client.48124 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi102-80356-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:41.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:07:41.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:07:41.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[41371]: pgmap v467: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:07:41.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:41.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:07:41.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:07:41.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/763872328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:41.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[29849]: from='client.48124 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:41.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/763872328' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:41.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[29849]: from='client.48124 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:41.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/763872328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi102-80356-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:41.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[29849]: from='client.48124 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi102-80356-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:41.649 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:07:41.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:07:41.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[29849]: pgmap v467: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:07:41.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:07:41.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:07:41.650 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:41 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:07:42.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:42 smithi146 ceph-mon[33158]: from='client.48124 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi102-80356-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:42.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:42 smithi146 ceph-mon[33158]: osdmap e333: 8 total, 8 up, 8 in 2024-09-11T08:07:42.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/763872328' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi102-80356-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:42.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:42 smithi146 ceph-mon[33158]: from='client.48124 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi102-80356-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:42.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:42 smithi102 ceph-mon[41371]: from='client.48124 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi102-80356-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:42.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:42 smithi102 ceph-mon[41371]: osdmap e333: 8 total, 8 up, 8 in 2024-09-11T08:07:42.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/763872328' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi102-80356-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:42.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:42 smithi102 ceph-mon[41371]: from='client.48124 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi102-80356-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:42.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:42 smithi102 ceph-mon[29849]: from='client.48124 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi102-80356-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:42.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:42 smithi102 ceph-mon[29849]: osdmap e333: 8 total, 8 up, 8 in 2024-09-11T08:07:42.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/763872328' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi102-80356-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:42.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:42 smithi102 ceph-mon[29849]: from='client.48124 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi102-80356-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:43.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:43 smithi146 ceph-mon[33158]: osdmap e334: 8 total, 8 up, 8 in 2024-09-11T08:07:43.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:43 smithi146 ceph-mon[33158]: pgmap v470: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:07:43.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:43 smithi146 ceph-mon[33158]: from='client.48124 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi102-80356-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi102-80356-64"}]': finished 2024-09-11T08:07:43.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:43 smithi146 ceph-mon[33158]: osdmap e335: 8 total, 8 up, 8 in 2024-09-11T08:07:43.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:43 smithi102 ceph-mon[41371]: osdmap e334: 8 total, 8 up, 8 in 2024-09-11T08:07:43.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:43 smithi102 ceph-mon[41371]: pgmap v470: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:07:43.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:43 smithi102 ceph-mon[41371]: from='client.48124 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi102-80356-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi102-80356-64"}]': finished 2024-09-11T08:07:43.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:43 smithi102 ceph-mon[41371]: osdmap e335: 8 total, 8 up, 8 in 2024-09-11T08:07:43.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:43 smithi102 ceph-mon[29849]: osdmap e334: 8 total, 8 up, 8 in 2024-09-11T08:07:43.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:43 smithi102 ceph-mon[29849]: pgmap v470: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:07:43.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:43 smithi102 ceph-mon[29849]: from='client.48124 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi102-80356-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi102-80356-64"}]': finished 2024-09-11T08:07:43.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:43 smithi102 ceph-mon[29849]: osdmap e335: 8 total, 8 up, 8 in 2024-09-11T08:07:44.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:07:44 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:07:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:07:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:45 smithi146 ceph-mon[33158]: osdmap e336: 8 total, 8 up, 8 in 2024-09-11T08:07:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:45 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:07:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:45 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:07:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:45 smithi146 ceph-mon[33158]: pgmap v473: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:45.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:45 smithi102 ceph-mon[41371]: osdmap e336: 8 total, 8 up, 8 in 2024-09-11T08:07:45.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:45 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:07:45.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:45 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:07:45.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:45 smithi102 ceph-mon[41371]: pgmap v473: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:45.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:45 smithi102 ceph-mon[29849]: osdmap e336: 8 total, 8 up, 8 in 2024-09-11T08:07:45.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:45 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:07:45.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:45 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:07:45.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:45 smithi102 ceph-mon[29849]: pgmap v473: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:46.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:46 smithi146 ceph-mon[33158]: osdmap e337: 8 total, 8 up, 8 in 2024-09-11T08:07:46.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:46 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/763872328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:46.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:46 smithi146 ceph-mon[33158]: from='client.48124 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:46.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:46 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:46.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:46 smithi102 ceph-mon[29849]: osdmap e337: 8 total, 8 up, 8 in 2024-09-11T08:07:46.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/763872328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:46.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:46 smithi102 ceph-mon[29849]: from='client.48124 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:46.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:46 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:46.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:46 smithi102 ceph-mon[41371]: osdmap e337: 8 total, 8 up, 8 in 2024-09-11T08:07:46.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:46 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/763872328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:46.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:46 smithi102 ceph-mon[41371]: from='client.48124 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:46.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:46 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:47.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:47 smithi102 ceph-mon[41371]: from='client.48124 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi102-80356-64"}]': finished 2024-09-11T08:07:47.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:47 smithi102 ceph-mon[41371]: osdmap e338: 8 total, 8 up, 8 in 2024-09-11T08:07:47.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/763872328' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:47.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:47 smithi102 ceph-mon[41371]: from='client.48124 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:47.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:47 smithi102 ceph-mon[41371]: pgmap v476: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:47.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:47 smithi102 ceph-mon[41371]: from='client.48124 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi102-80356-64"}]': finished 2024-09-11T08:07:47.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:47 smithi102 ceph-mon[41371]: osdmap e339: 8 total, 8 up, 8 in 2024-09-11T08:07:47.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:47 smithi102 ceph-mon[29849]: from='client.48124 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi102-80356-64"}]': finished 2024-09-11T08:07:47.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:47 smithi102 ceph-mon[29849]: osdmap e338: 8 total, 8 up, 8 in 2024-09-11T08:07:47.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/763872328' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:47.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:47 smithi102 ceph-mon[29849]: from='client.48124 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:47.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:47 smithi102 ceph-mon[29849]: pgmap v476: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:47.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:47 smithi102 ceph-mon[29849]: from='client.48124 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi102-80356-64"}]': finished 2024-09-11T08:07:47.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:47 smithi102 ceph-mon[29849]: osdmap e339: 8 total, 8 up, 8 in 2024-09-11T08:07:47.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:47 smithi146 ceph-mon[33158]: from='client.48124 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi102-80356-64"}]': finished 2024-09-11T08:07:47.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:47 smithi146 ceph-mon[33158]: osdmap e338: 8 total, 8 up, 8 in 2024-09-11T08:07:47.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/763872328' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:47.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:47 smithi146 ceph-mon[33158]: from='client.48124 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi102-80356-64"}]: dispatch 2024-09-11T08:07:47.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:47 smithi146 ceph-mon[33158]: pgmap v476: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:47.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:47 smithi146 ceph-mon[33158]: from='client.48124 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi102-80356-64"}]': finished 2024-09-11T08:07:47.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:47 smithi146 ceph-mon[33158]: osdmap e339: 8 total, 8 up, 8 in 2024-09-11T08:07:48.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi102-80356-65"}]: dispatch 2024-09-11T08:07:48.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi102-80356-65"}]: dispatch 2024-09-11T08:07:48.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi102-80356-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:48.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi102-80356-65"}]: dispatch 2024-09-11T08:07:48.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi102-80356-65"}]: dispatch 2024-09-11T08:07:48.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi102-80356-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:48.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi102-80356-65"}]: dispatch 2024-09-11T08:07:48.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi102-80356-65"}]: dispatch 2024-09-11T08:07:48.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi102-80356-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:49.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:49 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi102-80356-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:49.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:49 smithi102 ceph-mon[41371]: osdmap e340: 8 total, 8 up, 8 in 2024-09-11T08:07:49.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:49 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi102-80356-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi102-80356-65"}]: dispatch 2024-09-11T08:07:49.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:49 smithi102 ceph-mon[41371]: pgmap v479: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:49.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:49 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi102-80356-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:49.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:49 smithi102 ceph-mon[29849]: osdmap e340: 8 total, 8 up, 8 in 2024-09-11T08:07:49.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:49 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi102-80356-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi102-80356-65"}]: dispatch 2024-09-11T08:07:49.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:49 smithi102 ceph-mon[29849]: pgmap v479: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:49.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:07:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:07:49.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:49 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi102-80356-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:49.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:49 smithi146 ceph-mon[33158]: osdmap e340: 8 total, 8 up, 8 in 2024-09-11T08:07:49.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:49 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi102-80356-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi102-80356-65"}]: dispatch 2024-09-11T08:07:49.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:49 smithi146 ceph-mon[33158]: pgmap v479: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:50.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:07:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:07:49] "GET /metrics HTTP/1.1" 200 230961 "" "Prometheus/2.43.0" 2024-09-11T08:07:50.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:50 smithi102 ceph-mon[29849]: osdmap e341: 8 total, 8 up, 8 in 2024-09-11T08:07:50.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:50 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:50.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:50 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi102-80356-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi102-80356-65"}]': finished 2024-09-11T08:07:50.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:50 smithi102 ceph-mon[29849]: osdmap e342: 8 total, 8 up, 8 in 2024-09-11T08:07:50.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:50 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:50.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:50 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:50.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:50 smithi102 ceph-mon[41371]: osdmap e341: 8 total, 8 up, 8 in 2024-09-11T08:07:50.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:50 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:50.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:50 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi102-80356-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi102-80356-65"}]': finished 2024-09-11T08:07:50.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:50 smithi102 ceph-mon[41371]: osdmap e342: 8 total, 8 up, 8 in 2024-09-11T08:07:50.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:50 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:50.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:50 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:50.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:50 smithi146 ceph-mon[33158]: osdmap e341: 8 total, 8 up, 8 in 2024-09-11T08:07:50.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:50 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:07:50.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:50 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi102-80356-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi102-80356-65"}]': finished 2024-09-11T08:07:50.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:50 smithi146 ceph-mon[33158]: osdmap e342: 8 total, 8 up, 8 in 2024-09-11T08:07:50.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:50 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:50.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:50 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:51.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:51 smithi102 ceph-mon[41371]: pgmap v482: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:51.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:51 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:07:51.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:51 smithi102 ceph-mon[41371]: osdmap e343: 8 total, 8 up, 8 in 2024-09-11T08:07:51.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:51 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49"}]: dispatch 2024-09-11T08:07:51.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:51 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49"}]: dispatch 2024-09-11T08:07:51.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:51 smithi102 ceph-mon[29849]: pgmap v482: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:51.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:51 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:07:51.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:51 smithi102 ceph-mon[29849]: osdmap e343: 8 total, 8 up, 8 in 2024-09-11T08:07:51.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:51 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49"}]: dispatch 2024-09-11T08:07:51.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:51 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49"}]: dispatch 2024-09-11T08:07:51.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:51 smithi146 ceph-mon[33158]: pgmap v482: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:51.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:51 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:07:51.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:51 smithi146 ceph-mon[33158]: osdmap e343: 8 total, 8 up, 8 in 2024-09-11T08:07:51.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:51 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49"}]: dispatch 2024-09-11T08:07:51.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:51 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49"}]: dispatch 2024-09-11T08:07:52.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:52 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:52.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:52 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:07:52.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:52 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49"}]': finished 2024-09-11T08:07:52.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:52 smithi102 ceph-mon[41371]: osdmap e344: 8 total, 8 up, 8 in 2024-09-11T08:07:52.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:52 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi102-80356-65"}]: dispatch 2024-09-11T08:07:52.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:52 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:52.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:52 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:52.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:52 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49"}]: dispatch 2024-09-11T08:07:52.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:52 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49"}]: dispatch 2024-09-11T08:07:52.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:52 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:52.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:52 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:07:52.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:52 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49"}]': finished 2024-09-11T08:07:52.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:52 smithi102 ceph-mon[29849]: osdmap e344: 8 total, 8 up, 8 in 2024-09-11T08:07:52.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:52 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi102-80356-65"}]: dispatch 2024-09-11T08:07:52.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:52 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:52.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:52 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:52.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:52 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49"}]: dispatch 2024-09-11T08:07:52.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:52 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49"}]: dispatch 2024-09-11T08:07:52.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:52 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:52.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:52 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:07:52.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:52 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49"}]': finished 2024-09-11T08:07:52.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:52 smithi146 ceph-mon[33158]: osdmap e344: 8 total, 8 up, 8 in 2024-09-11T08:07:52.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:52 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi102-80356-65"}]: dispatch 2024-09-11T08:07:52.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:52 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:52.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:52 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:07:52.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:52 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49"}]: dispatch 2024-09-11T08:07:52.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:52 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-49"}]: dispatch 2024-09-11T08:07:53.198 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullP63:head 2024-09-11T08:07:53.198 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:5d165639:::164:head 2024-09-11T08:07:53.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:f43765fc:::165:head 2024-09-11T08:07:53.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:b4c720e9:::166:head 2024-09-11T08:07:53.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:e694b040:::167:head 2024-09-11T08:07:53.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:afa38db2:::168:head 2024-09-11T08:07:53.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:77ba9f53:::169:head 2024-09-11T08:07:53.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:87495034:::170:head 2024-09-11T08:07:53.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:7c96bf0e:::171:head 2024-09-11T08:07:53.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:dbe346cc:::172:head 2024-09-11T08:07:53.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:e943ec24:::173:head 2024-09-11T08:07:53.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:f97a9c0c:::174:head 2024-09-11T08:07:53.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:6f26e74d:::175:head 2024-09-11T08:07:53.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:4f95e106:::176:head 2024-09-11T08:07:53.199 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:0e6f2f8f:::177:head 2024-09-11T08:07:53.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:05db05f1:::178:head 2024-09-11T08:07:53.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:38a78d66:::179:head 2024-09-11T08:07:53.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:d095610b:::180:head 2024-09-11T08:07:53.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:a1a9d709:::181:head 2024-09-11T08:07:53.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:1e5d39db:::182:head 2024-09-11T08:07:53.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:f7df4fb9:::183:head 2024-09-11T08:07:53.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:03a7f161:::184:head 2024-09-11T08:07:53.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:ba70721e:::185:head 2024-09-11T08:07:53.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:28e5662d:::186:head 2024-09-11T08:07:53.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:973d52de:::187:head 2024-09-11T08:07:53.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:4303eb1c:::188:head 2024-09-11T08:07:53.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:b990b48e:::189:head 2024-09-11T08:07:53.200 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:29b8165b:::190:head 2024-09-11T08:07:53.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:3547f197:::191:head 2024-09-11T08:07:53.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:7e260936:::192:head 2024-09-11T08:07:53.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:1abec7b1:::193:head 2024-09-11T08:07:53.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:10fdda93:::194:head 2024-09-11T08:07:53.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:15817eea:::195:head 2024-09-11T08:07:53.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:770bab57:::196:head 2024-09-11T08:07:53.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:ed9e13e7:::197:head 2024-09-11T08:07:53.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:71471a8f:::198:head 2024-09-11T08:07:53.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: checking for 195:10fb1d02:::199:head 2024-09-11T08:07:53.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8016 ms) 2024-09-11T08:07:53.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-09-11T08:07:53.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042029,0 2024-09-11T08:07:53.201 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: first is 1726042029 2024-09-11T08:07:53.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042029,0 2024-09-11T08:07:53.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042029,0 2024-09-11T08:07:53.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042029,1726042031,1726042032,0 2024-09-11T08:07:53.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042029,1726042031,1726042032,0 2024-09-11T08:07:53.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042029,1726042031,1726042032,0 2024-09-11T08:07:53.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042029,1726042031,1726042032,1726042034,1726042035,0 2024-09-11T08:07:53.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042029,1726042031,1726042032,1726042034,1726042035,0 2024-09-11T08:07:53.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042029,1726042031,1726042032,1726042034,1726042035,0 2024-09-11T08:07:53.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042029,1726042031,1726042032,1726042034,1726042035,1726042037,1726042038,0 2024-09-11T08:07:53.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042029,1726042031,1726042032,1726042034,1726042035,1726042037,1726042038,0 2024-09-11T08:07:53.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042029,1726042031,1726042032,1726042034,1726042035,1726042037,1726042038,0 2024-09-11T08:07:53.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042032,1726042034,1726042035,1726042037,1726042038,1726042040,1726042041,0 2024-09-11T08:07:53.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: first now 1726042032, trimmed 2024-09-11T08:07:53.202 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (20323 ms) 2024-09-11T08:07:53.203 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-09-11T08:07:53.203 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: foo0 2024-09-11T08:07:53.203 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-09-11T08:07:53.203 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14050 ms) 2024-09-11T08:07:53.203 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-09-11T08:07:53.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:53 smithi102 ceph-mon[41371]: pgmap v485: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:53.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:53 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi102-80356-65"}]': finished 2024-09-11T08:07:53.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:53 smithi102 ceph-mon[41371]: osdmap e345: 8 total, 8 up, 8 in 2024-09-11T08:07:53.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:53 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi102-80356-65"}]: dispatch 2024-09-11T08:07:53.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:53 smithi102 ceph-mon[29849]: pgmap v485: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:53.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:53 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi102-80356-65"}]': finished 2024-09-11T08:07:53.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:53 smithi102 ceph-mon[29849]: osdmap e345: 8 total, 8 up, 8 in 2024-09-11T08:07:53.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:53 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi102-80356-65"}]: dispatch 2024-09-11T08:07:53.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:53 smithi146 ceph-mon[33158]: pgmap v485: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:07:53.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:53 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi102-80356-65"}]': finished 2024-09-11T08:07:53.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:53 smithi146 ceph-mon[33158]: osdmap e345: 8 total, 8 up, 8 in 2024-09-11T08:07:53.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:53 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi102-80356-65"}]: dispatch 2024-09-11T08:07:54.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:07:54 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:07:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:07:55.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi102-80356-65"}]': finished 2024-09-11T08:07:55.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:55 smithi102 ceph-mon[41371]: osdmap e346: 8 total, 8 up, 8 in 2024-09-11T08:07:55.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:55.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:55 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:55.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3791554891' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:55.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:55 smithi102 ceph-mon[41371]: from='client.47036 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:55.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3791554891' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:55.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:55 smithi102 ceph-mon[41371]: from='client.47036 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:55.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3791554891' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi102-80356-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:55.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:55 smithi102 ceph-mon[41371]: from='client.47036 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi102-80356-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:55.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:55 smithi102 ceph-mon[41371]: pgmap v488: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:55.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi102-80356-65"}]': finished 2024-09-11T08:07:55.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:55 smithi102 ceph-mon[29849]: osdmap e346: 8 total, 8 up, 8 in 2024-09-11T08:07:55.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:55.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:55 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:55.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3791554891' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:55.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:55 smithi102 ceph-mon[29849]: from='client.47036 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:55.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3791554891' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:55.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:55 smithi102 ceph-mon[29849]: from='client.47036 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:55.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3791554891' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi102-80356-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:55.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:55 smithi102 ceph-mon[29849]: from='client.47036 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi102-80356-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:55.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:55 smithi102 ceph-mon[29849]: pgmap v488: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:55.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2060522972' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi102-80356-65"}]': finished 2024-09-11T08:07:55.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:55 smithi146 ceph-mon[33158]: osdmap e346: 8 total, 8 up, 8 in 2024-09-11T08:07:55.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:55.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:55 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:07:55.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3791554891' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:55.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:55 smithi146 ceph-mon[33158]: from='client.47036 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:55.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3791554891' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:55.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:55 smithi146 ceph-mon[33158]: from='client.47036 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:55.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3791554891' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi102-80356-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:55.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:55 smithi146 ceph-mon[33158]: from='client.47036 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi102-80356-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:07:55.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:55 smithi146 ceph-mon[33158]: pgmap v488: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:07:56.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:56 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:07:56.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:56 smithi102 ceph-mon[41371]: from='client.47036 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi102-80356-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:56.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:56 smithi102 ceph-mon[41371]: osdmap e347: 8 total, 8 up, 8 in 2024-09-11T08:07:56.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:56 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3791554891' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi102-80356-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:56.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:56 smithi102 ceph-mon[41371]: from='client.47036 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi102-80356-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:56.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:56 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:56.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:56 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:56.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:56 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:07:56.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:56 smithi102 ceph-mon[29849]: from='client.47036 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi102-80356-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:56.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:56 smithi102 ceph-mon[29849]: osdmap e347: 8 total, 8 up, 8 in 2024-09-11T08:07:56.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:56 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3791554891' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi102-80356-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:56.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:56 smithi102 ceph-mon[29849]: from='client.47036 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi102-80356-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:56.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:56 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:56.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:56 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:56.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:56 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:07:56.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:56 smithi146 ceph-mon[33158]: from='client.47036 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi102-80356-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:07:56.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:56 smithi146 ceph-mon[33158]: osdmap e347: 8 total, 8 up, 8 in 2024-09-11T08:07:56.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:56 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3791554891' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi102-80356-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:56.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:56 smithi146 ceph-mon[33158]: from='client.47036 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi102-80356-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:56.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:56 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:56.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:56 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:07:57.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:57 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:07:57.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:57 smithi102 ceph-mon[29849]: osdmap e348: 8 total, 8 up, 8 in 2024-09-11T08:07:57.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:57 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-51"}]: dispatch 2024-09-11T08:07:57.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:57 smithi102 ceph-mon[29849]: pgmap v491: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:07:57.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:57 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-51"}]: dispatch 2024-09-11T08:07:57.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:57 smithi102 ceph-mon[29849]: from='client.47036 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi102-80356-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi102-80356-66"}]': finished 2024-09-11T08:07:57.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:57 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-51"}]': finished 2024-09-11T08:07:57.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:57 smithi102 ceph-mon[29849]: osdmap e349: 8 total, 8 up, 8 in 2024-09-11T08:07:57.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:57 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-51", "mode": "writeback"}]: dispatch 2024-09-11T08:07:57.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:57 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-51", "mode": "writeback"}]: dispatch 2024-09-11T08:07:57.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:57 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:07:57.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:57 smithi102 ceph-mon[41371]: osdmap e348: 8 total, 8 up, 8 in 2024-09-11T08:07:57.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:57 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-51"}]: dispatch 2024-09-11T08:07:57.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:57 smithi102 ceph-mon[41371]: pgmap v491: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:07:57.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:57 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-51"}]: dispatch 2024-09-11T08:07:57.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:57 smithi102 ceph-mon[41371]: from='client.47036 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi102-80356-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi102-80356-66"}]': finished 2024-09-11T08:07:57.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:57 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-51"}]': finished 2024-09-11T08:07:57.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:57 smithi102 ceph-mon[41371]: osdmap e349: 8 total, 8 up, 8 in 2024-09-11T08:07:57.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:57 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-51", "mode": "writeback"}]: dispatch 2024-09-11T08:07:57.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:57 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-51", "mode": "writeback"}]: dispatch 2024-09-11T08:07:57.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:57 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:07:57.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:57 smithi146 ceph-mon[33158]: osdmap e348: 8 total, 8 up, 8 in 2024-09-11T08:07:57.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:57 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-51"}]: dispatch 2024-09-11T08:07:57.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:57 smithi146 ceph-mon[33158]: pgmap v491: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:07:57.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:57 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-51"}]: dispatch 2024-09-11T08:07:57.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:57 smithi146 ceph-mon[33158]: from='client.47036 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi102-80356-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi102-80356-66"}]': finished 2024-09-11T08:07:57.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:57 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-51"}]': finished 2024-09-11T08:07:57.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:57 smithi146 ceph-mon[33158]: osdmap e349: 8 total, 8 up, 8 in 2024-09-11T08:07:57.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:57 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-51", "mode": "writeback"}]: dispatch 2024-09-11T08:07:57.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:57 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-51", "mode": "writeback"}]: dispatch 2024-09-11T08:07:58.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:58 smithi102 ceph-mon[41371]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:58.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:58 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:07:58.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:58 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-51", "mode": "writeback"}]': finished 2024-09-11T08:07:58.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:58 smithi102 ceph-mon[41371]: osdmap e350: 8 total, 8 up, 8 in 2024-09-11T08:07:58.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:58 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:07:58.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:58 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:07:58.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:58 smithi102 ceph-mon[29849]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:58.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:58 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:07:58.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:58 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-51", "mode": "writeback"}]': finished 2024-09-11T08:07:58.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:58 smithi102 ceph-mon[29849]: osdmap e350: 8 total, 8 up, 8 in 2024-09-11T08:07:58.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:58 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:07:58.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:58 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:07:58.936 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:58 smithi146 ceph-mon[33158]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:07:58.936 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:58 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:07:58.936 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:58 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-51", "mode": "writeback"}]': finished 2024-09-11T08:07:58.937 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:58 smithi146 ceph-mon[33158]: osdmap e350: 8 total, 8 up, 8 in 2024-09-11T08:07:58.937 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:58 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:07:58.937 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:58 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:07:59.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:59 smithi102 ceph-mon[41371]: pgmap v494: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:07:59.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:59 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:07:59.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:59 smithi102 ceph-mon[41371]: osdmap e351: 8 total, 8 up, 8 in 2024-09-11T08:07:59.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:59 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:07:59.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:59 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3791554891' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:59.850 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:59 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:07:59.850 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:59 smithi102 ceph-mon[41371]: from='client.47036 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:59.850 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:07:59 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:07:59.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:59 smithi102 ceph-mon[29849]: pgmap v494: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:07:59.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:59 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:07:59.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:59 smithi102 ceph-mon[29849]: osdmap e351: 8 total, 8 up, 8 in 2024-09-11T08:07:59.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:59 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:07:59.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:59 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3791554891' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:59.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:59 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:07:59.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:59 smithi102 ceph-mon[29849]: from='client.47036 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:59.851 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:07:59 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:07:59.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:07:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:07:59.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:59 smithi146 ceph-mon[33158]: pgmap v494: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:07:59.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:59 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:07:59.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:59 smithi146 ceph-mon[33158]: osdmap e351: 8 total, 8 up, 8 in 2024-09-11T08:07:59.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:59 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:07:59.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:59 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3791554891' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:59.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:59 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:07:59.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:59 smithi146 ceph-mon[33158]: from='client.47036 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:07:59.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:07:59 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:08:00.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:07:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:07:59] "GET /metrics HTTP/1.1" 200 228505 "" "Prometheus/2.43.0" 2024-09-11T08:08:00.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:00 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:00.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:00 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:08:00.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:00 smithi102 ceph-mon[29849]: from='client.47036 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi102-80356-66"}]': finished 2024-09-11T08:08:00.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:00 smithi102 ceph-mon[29849]: osdmap e352: 8 total, 8 up, 8 in 2024-09-11T08:08:00.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:00 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:08:00.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:00 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3791554891' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:08:00.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:00 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:08:00.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:00 smithi102 ceph-mon[29849]: from='client.47036 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:08:00.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:00 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:00.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:00 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:08:00.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:00 smithi102 ceph-mon[41371]: from='client.47036 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi102-80356-66"}]': finished 2024-09-11T08:08:00.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:00 smithi102 ceph-mon[41371]: osdmap e352: 8 total, 8 up, 8 in 2024-09-11T08:08:00.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:00 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:08:00.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:00 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3791554891' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:08:00.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:00 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:08:00.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:00 smithi102 ceph-mon[41371]: from='client.47036 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:08:00.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:00 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:00.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:00 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:08:00.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:00 smithi146 ceph-mon[33158]: from='client.47036 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi102-80356-66"}]': finished 2024-09-11T08:08:00.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:00 smithi146 ceph-mon[33158]: osdmap e352: 8 total, 8 up, 8 in 2024-09-11T08:08:00.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:00 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:08:00.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:00 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3791554891' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:08:00.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:00 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:08:00.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:00 smithi146 ceph-mon[33158]: from='client.47036 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi102-80356-66"}]: dispatch 2024-09-11T08:08:01.234 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ P2 2024-09-11T08:08:01.234 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3025 ms) 2024-09-11T08:08:01.234 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-09-11T08:08:01.234 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7048 ms) 2024-09-11T08:08:01.235 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-09-11T08:08:01.235 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7006 ms) 2024-09-11T08:08:01.235 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-09-11T08:08:01.235 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7030 ms) 2024-09-11T08:08:01.235 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-09-11T08:08:01.235 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7036 ms) 2024-09-11T08:08:01.235 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-09-11T08:08:01.235 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7031 ms) 2024-09-11T08:08:01.235 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-09-11T08:08:01.235 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7024 ms) 2024-09-11T08:08:01.235 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139297 ms total) 2024-09-11T08:08:01.235 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: 2024-09-11T08:08:01.235 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-09-11T08:08:01.236 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (293094 ms total) 2024-09-11T08:08:01.236 INFO:tasks.workunit.client.0.smithi102.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-09-11T08:08:01.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:01 smithi102 ceph-mon[29849]: pgmap v497: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-11T08:08:01.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:01 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:08:01.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:01 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:08:01.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:01 smithi102 ceph-mon[29849]: from='client.47036 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi102-80356-66"}]': finished 2024-09-11T08:08:01.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:01 smithi102 ceph-mon[29849]: osdmap e353: 8 total, 8 up, 8 in 2024-09-11T08:08:01.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:01 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:08:01.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:01 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:08:01.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:01 smithi102 ceph-mon[41371]: pgmap v497: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-11T08:08:01.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:01 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:08:01.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:01 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:08:01.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:01 smithi102 ceph-mon[41371]: from='client.47036 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi102-80356-66"}]': finished 2024-09-11T08:08:01.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:01 smithi102 ceph-mon[41371]: osdmap e353: 8 total, 8 up, 8 in 2024-09-11T08:08:01.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:01 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:08:01.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:01 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:08:01.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:01 smithi146 ceph-mon[33158]: pgmap v497: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-11T08:08:01.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:01 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:08:01.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:01 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:08:01.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:01 smithi146 ceph-mon[33158]: from='client.47036 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi102-80356-66"}]': finished 2024-09-11T08:08:01.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:01 smithi146 ceph-mon[33158]: osdmap e353: 8 total, 8 up, 8 in 2024-09-11T08:08:01.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:01 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:08:01.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:01 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:08:03.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:03 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:08:03.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:03 smithi102 ceph-mon[29849]: osdmap e354: 8 total, 8 up, 8 in 2024-09-11T08:08:03.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:03 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-11T08:08:03.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:03 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-11T08:08:03.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:03 smithi102 ceph-mon[29849]: pgmap v500: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-11T08:08:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:03 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:08:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:03 smithi102 ceph-mon[41371]: osdmap e354: 8 total, 8 up, 8 in 2024-09-11T08:08:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:03 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-11T08:08:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:03 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-11T08:08:03.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:03 smithi102 ceph-mon[41371]: pgmap v500: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-11T08:08:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:03 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:08:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:03 smithi146 ceph-mon[33158]: osdmap e354: 8 total, 8 up, 8 in 2024-09-11T08:08:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:03 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-11T08:08:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:03 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-11T08:08:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:03 smithi146 ceph-mon[33158]: pgmap v500: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-11T08:08:04.586 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:04 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "target_max_objects","val": "1"}]': finished 2024-09-11T08:08:04.586 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:04 smithi146 ceph-mon[33158]: osdmap e355: 8 total, 8 up, 8 in 2024-09-11T08:08:04.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:04 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "target_max_objects","val": "1"}]': finished 2024-09-11T08:08:04.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:04 smithi102 ceph-mon[29849]: osdmap e355: 8 total, 8 up, 8 in 2024-09-11T08:08:04.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:04 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-51","var": "target_max_objects","val": "1"}]': finished 2024-09-11T08:08:04.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:04 smithi102 ceph-mon[41371]: osdmap e355: 8 total, 8 up, 8 in 2024-09-11T08:08:04.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:08:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:08:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:08:05.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:05 smithi102 ceph-mon[29849]: pgmap v502: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-11T08:08:05.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:05 smithi102 ceph-mon[41371]: pgmap v502: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-11T08:08:05.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:05 smithi146 ceph-mon[33158]: pgmap v502: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-11T08:08:06.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:06 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-11T08:08:06.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:06 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-11T08:08:06.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:06 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-11T08:08:07.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:07 smithi102 ceph-mon[29849]: pgmap v503: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:08:07.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:07 smithi102 ceph-mon[41371]: pgmap v503: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:08:07.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:07 smithi146 ceph-mon[33158]: pgmap v503: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:08:09.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:09 smithi102 ceph-mon[41371]: pgmap v504: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 708 B/s rd, 0 B/s wr, 0 op/s 2024-09-11T08:08:09.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:09 smithi102 ceph-mon[29849]: pgmap v504: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 708 B/s rd, 0 B/s wr, 0 op/s 2024-09-11T08:08:09.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:09 smithi146 ceph-mon[33158]: pgmap v504: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 708 B/s rd, 0 B/s wr, 0 op/s 2024-09-11T08:08:09.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:08:09 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:08:10.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:08:09 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:08:09] "GET /metrics HTTP/1.1" 200 228505 "" "Prometheus/2.43.0" 2024-09-11T08:08:10.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:10 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:10.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:10 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:10.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:10 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:11.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:11 smithi102 ceph-mon[29849]: pgmap v505: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:08:11.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:11 smithi102 ceph-mon[41371]: pgmap v505: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:08:11.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:11 smithi146 ceph-mon[33158]: pgmap v505: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:08:13.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:13 smithi102 ceph-mon[29849]: pgmap v506: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:08:13.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:13.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:13 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:13.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:13 smithi102 ceph-mon[41371]: pgmap v506: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:08:13.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:13.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:13 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:13.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:13 smithi146 ceph-mon[33158]: pgmap v506: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:08:13.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:13.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:13 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:14.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:14 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:08:14.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:14 smithi102 ceph-mon[29849]: osdmap e356: 8 total, 8 up, 8 in 2024-09-11T08:08:14.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51"}]: dispatch 2024-09-11T08:08:14.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:14 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51"}]: dispatch 2024-09-11T08:08:14.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:14 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:08:14.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:14 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:08:14.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:14 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:08:14.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:14 smithi102 ceph-mon[41371]: osdmap e356: 8 total, 8 up, 8 in 2024-09-11T08:08:14.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51"}]: dispatch 2024-09-11T08:08:14.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:14 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51"}]: dispatch 2024-09-11T08:08:14.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:14 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:08:14.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:14 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:08:14.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:08:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:08:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:08:14.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:14 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:08:14.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:14 smithi146 ceph-mon[33158]: osdmap e356: 8 total, 8 up, 8 in 2024-09-11T08:08:14.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51"}]: dispatch 2024-09-11T08:08:14.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:14 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51"}]: dispatch 2024-09-11T08:08:14.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:14 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:08:14.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:14 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:08:16.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:15 smithi102 ceph-mon[41371]: pgmap v508: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:08:16.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:15 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51"}]': finished 2024-09-11T08:08:16.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:15 smithi102 ceph-mon[41371]: osdmap e357: 8 total, 8 up, 8 in 2024-09-11T08:08:16.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:15 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:16.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:15 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:16.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:15 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51"}]: dispatch 2024-09-11T08:08:16.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:15 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51"}]: dispatch 2024-09-11T08:08:16.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:15 smithi102 ceph-mon[29849]: pgmap v508: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:08:16.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:15 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51"}]': finished 2024-09-11T08:08:16.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:15 smithi102 ceph-mon[29849]: osdmap e357: 8 total, 8 up, 8 in 2024-09-11T08:08:16.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:15 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:16.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:15 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:16.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:15 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51"}]: dispatch 2024-09-11T08:08:16.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:15 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51"}]: dispatch 2024-09-11T08:08:16.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:15 smithi146 ceph-mon[33158]: pgmap v508: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:08:16.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:15 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51"}]': finished 2024-09-11T08:08:16.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:15 smithi146 ceph-mon[33158]: osdmap e357: 8 total, 8 up, 8 in 2024-09-11T08:08:16.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:15 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:16.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:15 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:16.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:15 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51"}]: dispatch 2024-09-11T08:08:16.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:15 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-51"}]: dispatch 2024-09-11T08:08:17.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:16 smithi102 ceph-mon[41371]: osdmap e358: 8 total, 8 up, 8 in 2024-09-11T08:08:17.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:16 smithi102 ceph-mon[41371]: osdmap e359: 8 total, 8 up, 8 in 2024-09-11T08:08:17.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:17.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:16 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:17.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:16 smithi102 ceph-mon[29849]: osdmap e358: 8 total, 8 up, 8 in 2024-09-11T08:08:17.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:16 smithi102 ceph-mon[29849]: osdmap e359: 8 total, 8 up, 8 in 2024-09-11T08:08:17.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:17.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:16 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:17.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:16 smithi146 ceph-mon[33158]: osdmap e358: 8 total, 8 up, 8 in 2024-09-11T08:08:17.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:16 smithi146 ceph-mon[33158]: osdmap e359: 8 total, 8 up, 8 in 2024-09-11T08:08:17.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:17.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:16 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:18.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:17 smithi102 ceph-mon[41371]: pgmap v511: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-11T08:08:18.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:17 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-11T08:08:18.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:17 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:08:18.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:17 smithi102 ceph-mon[41371]: osdmap e360: 8 total, 8 up, 8 in 2024-09-11T08:08:18.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:18.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:17 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:18.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-53"}]: dispatch 2024-09-11T08:08:18.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:17 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-53"}]: dispatch 2024-09-11T08:08:18.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:17 smithi102 ceph-mon[29849]: pgmap v511: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-11T08:08:18.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:17 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-11T08:08:18.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:17 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:08:18.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:17 smithi102 ceph-mon[29849]: osdmap e360: 8 total, 8 up, 8 in 2024-09-11T08:08:18.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:18.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:17 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:18.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-53"}]: dispatch 2024-09-11T08:08:18.148 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:17 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-53"}]: dispatch 2024-09-11T08:08:18.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:17 smithi146 ceph-mon[33158]: pgmap v511: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-11T08:08:18.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:17 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-11T08:08:18.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:17 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:08:18.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:17 smithi146 ceph-mon[33158]: osdmap e360: 8 total, 8 up, 8 in 2024-09-11T08:08:18.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:18.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:17 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:18.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-53"}]: dispatch 2024-09-11T08:08:18.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:17 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-53"}]: dispatch 2024-09-11T08:08:19.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:19 smithi102 ceph-mon[41371]: pgmap v514: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-11T08:08:19.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:19 smithi102 ceph-mon[41371]: osdmap e361: 8 total, 8 up, 8 in 2024-09-11T08:08:19.849 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:19 smithi102 ceph-mon[29849]: pgmap v514: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-11T08:08:19.849 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:19 smithi102 ceph-mon[29849]: osdmap e361: 8 total, 8 up, 8 in 2024-09-11T08:08:19.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:19 smithi146 ceph-mon[33158]: pgmap v514: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-11T08:08:19.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:19 smithi146 ceph-mon[33158]: osdmap e361: 8 total, 8 up, 8 in 2024-09-11T08:08:19.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:08:19 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:08:20.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:08:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:08:19] "GET /metrics HTTP/1.1" 200 228616 "" "Prometheus/2.43.0" 2024-09-11T08:08:20.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:20 smithi102 ceph-mon[29849]: osdmap e362: 8 total, 8 up, 8 in 2024-09-11T08:08:20.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:20.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:20 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:20.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:20 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:20.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:20 smithi102 ceph-mon[41371]: osdmap e362: 8 total, 8 up, 8 in 2024-09-11T08:08:20.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:20.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:20 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:20.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:20 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:20.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:20 smithi146 ceph-mon[33158]: osdmap e362: 8 total, 8 up, 8 in 2024-09-11T08:08:20.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:20.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:20 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:20.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:20 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:21.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:21 smithi102 ceph-mon[29849]: pgmap v517: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:08:21.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:21 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:08:21.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:21 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:08:21.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:21 smithi102 ceph-mon[29849]: osdmap e363: 8 total, 8 up, 8 in 2024-09-11T08:08:21.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:21 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:08:21.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:21 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:21.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:21 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:21.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:21 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-55"}]: dispatch 2024-09-11T08:08:21.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:21 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-55"}]: dispatch 2024-09-11T08:08:21.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:21 smithi102 ceph-mon[41371]: pgmap v517: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:08:21.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:21 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:08:21.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:21 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:08:21.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:21 smithi102 ceph-mon[41371]: osdmap e363: 8 total, 8 up, 8 in 2024-09-11T08:08:21.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:21 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:08:21.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:21 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:21.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:21 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:21.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:21 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-55"}]: dispatch 2024-09-11T08:08:21.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:21 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-55"}]: dispatch 2024-09-11T08:08:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:21 smithi146 ceph-mon[33158]: pgmap v517: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:08:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:21 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:08:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:21 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:08:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:21 smithi146 ceph-mon[33158]: osdmap e363: 8 total, 8 up, 8 in 2024-09-11T08:08:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:21 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:08:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:21 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:21 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:21 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-55"}]: dispatch 2024-09-11T08:08:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:21 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-55"}]: dispatch 2024-09-11T08:08:22.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:22 smithi102 ceph-mon[41371]: osdmap e364: 8 total, 8 up, 8 in 2024-09-11T08:08:22.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:22 smithi102 ceph-mon[29849]: osdmap e364: 8 total, 8 up, 8 in 2024-09-11T08:08:22.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:22 smithi146 ceph-mon[33158]: osdmap e364: 8 total, 8 up, 8 in 2024-09-11T08:08:23.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:23 smithi102 ceph-mon[41371]: pgmap v520: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:08:23.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:23 smithi102 ceph-mon[41371]: osdmap e365: 8 total, 8 up, 8 in 2024-09-11T08:08:23.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:23.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:23 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:23.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:23 smithi102 ceph-mon[29849]: pgmap v520: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:08:23.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:23 smithi102 ceph-mon[29849]: osdmap e365: 8 total, 8 up, 8 in 2024-09-11T08:08:23.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:23.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:23 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:23.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:23 smithi146 ceph-mon[33158]: pgmap v520: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:08:23.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:23 smithi146 ceph-mon[33158]: osdmap e365: 8 total, 8 up, 8 in 2024-09-11T08:08:23.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:23.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:23 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:24.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:24 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:08:24.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:24 smithi102 ceph-mon[29849]: osdmap e366: 8 total, 8 up, 8 in 2024-09-11T08:08:24.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:24.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:24 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:24.898 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-57"}]: dispatch 2024-09-11T08:08:24.899 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:24 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-57"}]: dispatch 2024-09-11T08:08:24.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:24 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:08:24.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:24 smithi102 ceph-mon[41371]: osdmap e366: 8 total, 8 up, 8 in 2024-09-11T08:08:24.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:24.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:24 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:24.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-57"}]: dispatch 2024-09-11T08:08:24.899 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:24 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-57"}]: dispatch 2024-09-11T08:08:24.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:08:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:08:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:08:24.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:24 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:08:24.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:24 smithi146 ceph-mon[33158]: osdmap e366: 8 total, 8 up, 8 in 2024-09-11T08:08:24.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:24.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:24 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:24.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-57"}]: dispatch 2024-09-11T08:08:24.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:24 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-57"}]: dispatch 2024-09-11T08:08:25.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:25 smithi102 ceph-mon[29849]: pgmap v523: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:08:25.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:25 smithi102 ceph-mon[29849]: osdmap e367: 8 total, 8 up, 8 in 2024-09-11T08:08:25.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:25 smithi102 ceph-mon[41371]: pgmap v523: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:08:25.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:25 smithi102 ceph-mon[41371]: osdmap e367: 8 total, 8 up, 8 in 2024-09-11T08:08:25.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:25 smithi146 ceph-mon[33158]: pgmap v523: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:08:25.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:25 smithi146 ceph-mon[33158]: osdmap e367: 8 total, 8 up, 8 in 2024-09-11T08:08:26.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:26 smithi102 ceph-mon[29849]: osdmap e368: 8 total, 8 up, 8 in 2024-09-11T08:08:26.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:26.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:26 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:26.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:26 smithi102 ceph-mon[41371]: osdmap e368: 8 total, 8 up, 8 in 2024-09-11T08:08:26.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:26.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:26 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:26.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:26 smithi146 ceph-mon[33158]: osdmap e368: 8 total, 8 up, 8 in 2024-09-11T08:08:26.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:26.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:26 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:27.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:27 smithi102 ceph-mon[29849]: pgmap v526: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:08:27.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:27 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:08:27.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:27 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:08:27.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:27 smithi102 ceph-mon[29849]: osdmap e369: 8 total, 8 up, 8 in 2024-09-11T08:08:27.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:08:27.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:27.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:27 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:27.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-59"}]: dispatch 2024-09-11T08:08:27.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:27 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-59"}]: dispatch 2024-09-11T08:08:27.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:27 smithi102 ceph-mon[41371]: pgmap v526: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:08:27.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:27 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:08:27.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:27 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:08:27.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:27 smithi102 ceph-mon[41371]: osdmap e369: 8 total, 8 up, 8 in 2024-09-11T08:08:27.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:08:27.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:27.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:27 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:27.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-59"}]: dispatch 2024-09-11T08:08:27.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:27 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-59"}]: dispatch 2024-09-11T08:08:27.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:27 smithi146 ceph-mon[33158]: pgmap v526: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:08:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:27 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:08:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:27 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:08:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:27 smithi146 ceph-mon[33158]: osdmap e369: 8 total, 8 up, 8 in 2024-09-11T08:08:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:08:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:27 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-59"}]: dispatch 2024-09-11T08:08:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:27 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-59"}]: dispatch 2024-09-11T08:08:28.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:28 smithi102 ceph-mon[29849]: osdmap e370: 8 total, 8 up, 8 in 2024-09-11T08:08:28.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:28 smithi102 ceph-mon[41371]: osdmap e370: 8 total, 8 up, 8 in 2024-09-11T08:08:28.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:28 smithi146 ceph-mon[33158]: osdmap e370: 8 total, 8 up, 8 in 2024-09-11T08:08:29.896 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:08:29 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:08:29] "GET /metrics HTTP/1.1" 200 228577 "" "Prometheus/2.43.0" 2024-09-11T08:08:29.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:29 smithi102 ceph-mon[29849]: pgmap v529: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:08:29.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:29 smithi102 ceph-mon[29849]: osdmap e371: 8 total, 8 up, 8 in 2024-09-11T08:08:29.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:29.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:29 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:29.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:29 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:08:29.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:29 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:08:29.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:29 smithi102 ceph-mon[41371]: pgmap v529: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:08:29.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:29 smithi102 ceph-mon[41371]: osdmap e371: 8 total, 8 up, 8 in 2024-09-11T08:08:29.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:29.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:29 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:29.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:29 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:08:29.898 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:29 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:08:29.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:08:29 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:08:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:29 smithi146 ceph-mon[33158]: pgmap v529: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:08:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:29 smithi146 ceph-mon[33158]: osdmap e371: 8 total, 8 up, 8 in 2024-09-11T08:08:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:29.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:29 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:29.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:29 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:08:29.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:29 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:08:30.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:30 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:08:30.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:30 smithi102 ceph-mon[29849]: osdmap e372: 8 total, 8 up, 8 in 2024-09-11T08:08:30.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:30 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:08:30.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:30 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:08:30.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:30 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:08:30.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:30 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:30.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:30 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:08:30.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:30 smithi102 ceph-mon[41371]: osdmap e372: 8 total, 8 up, 8 in 2024-09-11T08:08:30.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:30 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:08:30.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:30 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:08:30.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:30 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:08:30.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:30 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:30.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:30 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:08:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:30 smithi146 ceph-mon[33158]: osdmap e372: 8 total, 8 up, 8 in 2024-09-11T08:08:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:30 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:08:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:30 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:08:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:30 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:08:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:30 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:32.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:31 smithi102 ceph-mon[29849]: pgmap v532: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:08:32.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:31 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:08:32.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:31 smithi102 ceph-mon[29849]: osdmap e373: 8 total, 8 up, 8 in 2024-09-11T08:08:32.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:31 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:32.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:31 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:32.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:31 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-61"}]: dispatch 2024-09-11T08:08:32.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:31 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-61"}]: dispatch 2024-09-11T08:08:32.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:31 smithi102 ceph-mon[41371]: pgmap v532: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:08:32.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:31 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:08:32.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:31 smithi102 ceph-mon[41371]: osdmap e373: 8 total, 8 up, 8 in 2024-09-11T08:08:32.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:31 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:32.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:31 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:32.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:31 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-61"}]: dispatch 2024-09-11T08:08:32.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:31 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-61"}]: dispatch 2024-09-11T08:08:32.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:31 smithi146 ceph-mon[33158]: pgmap v532: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:08:32.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:31 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:08:32.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:31 smithi146 ceph-mon[33158]: osdmap e373: 8 total, 8 up, 8 in 2024-09-11T08:08:32.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:31 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:32.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:31 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:08:32.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:31 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-61"}]: dispatch 2024-09-11T08:08:32.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:31 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-61"}]: dispatch 2024-09-11T08:08:33.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:32 smithi102 ceph-mon[29849]: osdmap e374: 8 total, 8 up, 8 in 2024-09-11T08:08:33.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:32 smithi102 ceph-mon[29849]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:08:33.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:32 smithi102 ceph-mon[29849]: osdmap e375: 8 total, 8 up, 8 in 2024-09-11T08:08:33.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:33.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:32 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:33.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:32 smithi102 ceph-mon[41371]: osdmap e374: 8 total, 8 up, 8 in 2024-09-11T08:08:33.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:32 smithi102 ceph-mon[41371]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:08:33.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:32 smithi102 ceph-mon[41371]: osdmap e375: 8 total, 8 up, 8 in 2024-09-11T08:08:33.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:33.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:32 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:32 smithi146 ceph-mon[33158]: osdmap e374: 8 total, 8 up, 8 in 2024-09-11T08:08:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:32 smithi146 ceph-mon[33158]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:08:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:32 smithi146 ceph-mon[33158]: osdmap e375: 8 total, 8 up, 8 in 2024-09-11T08:08:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:32 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:08:34.149 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:33 smithi102 ceph-mon[29849]: pgmap v535: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:08:34.150 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:33 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:08:34.150 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:33 smithi102 ceph-mon[29849]: osdmap e376: 8 total, 8 up, 8 in 2024-09-11T08:08:34.150 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:08:34.150 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:08:34.150 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:33 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:08:34.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:33 smithi102 ceph-mon[41371]: pgmap v535: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:08:34.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:33 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:08:34.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:33 smithi102 ceph-mon[41371]: osdmap e376: 8 total, 8 up, 8 in 2024-09-11T08:08:34.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:08:34.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:08:34.151 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:33 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:08:34.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:33 smithi146 ceph-mon[33158]: pgmap v535: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T08:08:34.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:33 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:08:34.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:33 smithi146 ceph-mon[33158]: osdmap e376: 8 total, 8 up, 8 in 2024-09-11T08:08:34.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:08:34.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:08:34.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:33 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:08:34.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:08:34 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:08:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:08:35.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:35 smithi102 ceph-mon[29849]: pgmap v538: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:08:35.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:35 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:08:35.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:35 smithi102 ceph-mon[29849]: osdmap e377: 8 total, 8 up, 8 in 2024-09-11T08:08:35.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:35 smithi102 ceph-mon[41371]: pgmap v538: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:08:35.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:35 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:08:35.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:35 smithi102 ceph-mon[41371]: osdmap e377: 8 total, 8 up, 8 in 2024-09-11T08:08:35.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:35 smithi146 ceph-mon[33158]: pgmap v538: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:08:35.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:35 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:08:35.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:35 smithi146 ceph-mon[33158]: osdmap e377: 8 total, 8 up, 8 in 2024-09-11T08:08:36.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:36 smithi102 ceph-mon[29849]: osdmap e378: 8 total, 8 up, 8 in 2024-09-11T08:08:36.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:36 smithi102 ceph-mon[41371]: osdmap e378: 8 total, 8 up, 8 in 2024-09-11T08:08:36.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:36 smithi146 ceph-mon[33158]: osdmap e378: 8 total, 8 up, 8 in 2024-09-11T08:08:37.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:37 smithi146 ceph-mon[33158]: pgmap v541: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-11T08:08:37.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:37 smithi146 ceph-mon[33158]: osdmap e379: 8 total, 8 up, 8 in 2024-09-11T08:08:38.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:37 smithi102 ceph-mon[29849]: pgmap v541: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-11T08:08:38.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:37 smithi102 ceph-mon[29849]: osdmap e379: 8 total, 8 up, 8 in 2024-09-11T08:08:38.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:37 smithi102 ceph-mon[41371]: pgmap v541: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-11T08:08:38.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:37 smithi102 ceph-mon[41371]: osdmap e379: 8 total, 8 up, 8 in 2024-09-11T08:08:38.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:38 smithi146 ceph-mon[33158]: osdmap e380: 8 total, 8 up, 8 in 2024-09-11T08:08:39.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:38 smithi102 ceph-mon[29849]: osdmap e380: 8 total, 8 up, 8 in 2024-09-11T08:08:39.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:38 smithi102 ceph-mon[41371]: osdmap e380: 8 total, 8 up, 8 in 2024-09-11T08:08:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:39 smithi146 ceph-mon[33158]: pgmap v544: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-11T08:08:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:39 smithi146 ceph-mon[33158]: osdmap e381: 8 total, 8 up, 8 in 2024-09-11T08:08:39.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:08:39 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:08:40.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:08:39 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:08:39] "GET /metrics HTTP/1.1" 200 228577 "" "Prometheus/2.43.0" 2024-09-11T08:08:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:39 smithi102 ceph-mon[41371]: pgmap v544: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-11T08:08:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:39 smithi102 ceph-mon[41371]: osdmap e381: 8 total, 8 up, 8 in 2024-09-11T08:08:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:39 smithi102 ceph-mon[29849]: pgmap v544: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-11T08:08:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:39 smithi102 ceph-mon[29849]: osdmap e381: 8 total, 8 up, 8 in 2024-09-11T08:08:40.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:40 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:41.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:40 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:41.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:40 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:41.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:41 smithi146 ceph-mon[33158]: pgmap v546: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-11T08:08:41.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:41 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:41.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:41 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:41.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:41 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:41.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:41 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:08:42.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:41 smithi102 ceph-mon[29849]: pgmap v546: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-11T08:08:42.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:41 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:42.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:41 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:42.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:41 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:42.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:41 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:08:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:41 smithi102 ceph-mon[41371]: pgmap v546: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-11T08:08:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:41 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:08:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:41 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:08:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:41 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:08:42.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:41 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:08:43.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:42 smithi146 ceph-mon[33158]: osdmap e382: 8 total, 8 up, 8 in 2024-09-11T08:08:43.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:42 smithi102 ceph-mon[29849]: osdmap e382: 8 total, 8 up, 8 in 2024-09-11T08:08:43.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:42 smithi102 ceph-mon[41371]: osdmap e382: 8 total, 8 up, 8 in 2024-09-11T08:08:44.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:43 smithi146 ceph-mon[33158]: pgmap v548: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-11T08:08:44.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:43 smithi102 ceph-mon[41371]: pgmap v548: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-11T08:08:44.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:43 smithi102 ceph-mon[29849]: pgmap v548: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-11T08:08:44.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:08:44 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:08:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:08:45.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:44 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:08:45.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:44 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:08:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:44 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:08:46.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:45 smithi102 ceph-mon[29849]: pgmap v549: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 901 B/s wr, 3 op/s 2024-09-11T08:08:46.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:45 smithi102 ceph-mon[41371]: pgmap v549: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 901 B/s wr, 3 op/s 2024-09-11T08:08:46.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:45 smithi146 ceph-mon[33158]: pgmap v549: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 901 B/s wr, 3 op/s 2024-09-11T08:08:48.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:47 smithi102 ceph-mon[29849]: pgmap v550: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-11T08:08:48.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:47 smithi102 ceph-mon[41371]: pgmap v550: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-11T08:08:48.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:47 smithi146 ceph-mon[33158]: pgmap v550: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-11T08:08:50.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:08:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:08:49] "GET /metrics HTTP/1.1" 200 228625 "" "Prometheus/2.43.0" 2024-09-11T08:08:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:49 smithi102 ceph-mon[29849]: pgmap v551: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 626 B/s wr, 2 op/s 2024-09-11T08:08:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:49 smithi102 ceph-mon[29849]: osdmap e383: 8 total, 8 up, 8 in 2024-09-11T08:08:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:49 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:50.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:49 smithi102 ceph-mon[41371]: pgmap v551: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 626 B/s wr, 2 op/s 2024-09-11T08:08:50.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:49 smithi102 ceph-mon[41371]: osdmap e383: 8 total, 8 up, 8 in 2024-09-11T08:08:50.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:49 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:49 smithi146 ceph-mon[33158]: pgmap v551: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 626 B/s wr, 2 op/s 2024-09-11T08:08:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:49 smithi146 ceph-mon[33158]: osdmap e383: 8 total, 8 up, 8 in 2024-09-11T08:08:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:49 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:08:50.184 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:08:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:08:51.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:50 smithi102 ceph-mon[41371]: pgmap v553: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:08:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:50 smithi102 ceph-mon[29849]: pgmap v553: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:08:51.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:50 smithi146 ceph-mon[33158]: pgmap v553: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:08:53.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:52 smithi146 ceph-mon[33158]: osdmap e384: 8 total, 8 up, 8 in 2024-09-11T08:08:53.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:52 smithi102 ceph-mon[29849]: osdmap e384: 8 total, 8 up, 8 in 2024-09-11T08:08:53.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:52 smithi102 ceph-mon[41371]: osdmap e384: 8 total, 8 up, 8 in 2024-09-11T08:08:54.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:53 smithi146 ceph-mon[33158]: pgmap v555: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:08:54.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:53 smithi102 ceph-mon[29849]: pgmap v555: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:08:54.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:53 smithi102 ceph-mon[41371]: pgmap v555: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:08:54.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:08:54 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:08:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:08:56.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:55 smithi102 ceph-mon[41371]: pgmap v556: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:08:56.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:55 smithi102 ceph-mon[29849]: pgmap v556: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:08:56.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:55 smithi146 ceph-mon[33158]: pgmap v556: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:08:58.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:57 smithi102 ceph-mon[41371]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:08:58.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:57 smithi102 ceph-mon[29849]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:08:58.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:57 smithi146 ceph-mon[33158]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:09:00.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:08:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:08:59] "GET /metrics HTTP/1.1" 200 228612 "" "Prometheus/2.43.0" 2024-09-11T08:09:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:59 smithi102 ceph-mon[29849]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:09:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:08:59 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:00.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:59 smithi102 ceph-mon[41371]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:09:00.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:08:59 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:00.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:59 smithi146 ceph-mon[33158]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:09:00.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:08:59 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:00.184 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:08:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:09:01.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:00 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:01.602 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:00 smithi102 ceph-mon[29849]: osdmap e385: 8 total, 8 up, 8 in 2024-09-11T08:09:01.602 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:00 smithi102 ceph-mon[29849]: pgmap v560: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:09:01.602 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:00 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:01.603 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:00 smithi102 ceph-mon[41371]: osdmap e385: 8 total, 8 up, 8 in 2024-09-11T08:09:01.603 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:00 smithi102 ceph-mon[41371]: pgmap v560: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:09:01.603 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:00 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:01.603 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:00 smithi146 ceph-mon[33158]: osdmap e385: 8 total, 8 up, 8 in 2024-09-11T08:09:01.603 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:00 smithi146 ceph-mon[33158]: pgmap v560: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:09:03.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:03 smithi102 ceph-mon[41371]: pgmap v561: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:09:03.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:03 smithi102 ceph-mon[29849]: pgmap v561: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:09:03.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:03 smithi146 ceph-mon[33158]: pgmap v561: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:09:04.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:09:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:09:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:09:05.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:05 smithi102 ceph-mon[41371]: pgmap v562: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:09:05.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:05 smithi102 ceph-mon[29849]: pgmap v562: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:09:05.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:05 smithi146 ceph-mon[33158]: pgmap v562: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:09:08.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:07 smithi146 ceph-mon[33158]: pgmap v563: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:09:08.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:07 smithi146 ceph-mon[33158]: osdmap e386: 8 total, 8 up, 8 in 2024-09-11T08:09:08.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:07 smithi102 ceph-mon[41371]: pgmap v563: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:09:08.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:07 smithi102 ceph-mon[41371]: osdmap e386: 8 total, 8 up, 8 in 2024-09-11T08:09:08.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:07 smithi102 ceph-mon[29849]: pgmap v563: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:09:08.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:07 smithi102 ceph-mon[29849]: osdmap e386: 8 total, 8 up, 8 in 2024-09-11T08:09:10.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:09 smithi102 ceph-mon[41371]: pgmap v565: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:09:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:09 smithi102 ceph-mon[29849]: pgmap v565: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:09:10.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:09:09 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:09:09] "GET /metrics HTTP/1.1" 200 228612 "" "Prometheus/2.43.0" 2024-09-11T08:09:10.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:09 smithi146 ceph-mon[33158]: pgmap v565: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-11T08:09:10.184 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:09:09 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:09:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:10 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:10 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-63"}]: dispatch 2024-09-11T08:09:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:10 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-63"}]: dispatch 2024-09-11T08:09:11.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:10 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:11.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:11.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:10 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:11.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-63"}]: dispatch 2024-09-11T08:09:11.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:10 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-63"}]: dispatch 2024-09-11T08:09:11.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:10 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:11.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:11.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:10 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:11.436 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-63"}]: dispatch 2024-09-11T08:09:11.437 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:10 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-63"}]: dispatch 2024-09-11T08:09:12.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:11 smithi102 ceph-mon[29849]: pgmap v566: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:09:12.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:11 smithi102 ceph-mon[29849]: osdmap e387: 8 total, 8 up, 8 in 2024-09-11T08:09:12.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:11 smithi102 ceph-mon[41371]: pgmap v566: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:09:12.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:11 smithi102 ceph-mon[41371]: osdmap e387: 8 total, 8 up, 8 in 2024-09-11T08:09:12.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:11 smithi146 ceph-mon[33158]: pgmap v566: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:09:12.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:11 smithi146 ceph-mon[33158]: osdmap e387: 8 total, 8 up, 8 in 2024-09-11T08:09:13.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:12 smithi102 ceph-mon[29849]: osdmap e388: 8 total, 8 up, 8 in 2024-09-11T08:09:13.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:13.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:12 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:13.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:12 smithi102 ceph-mon[41371]: osdmap e388: 8 total, 8 up, 8 in 2024-09-11T08:09:13.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:13.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:12 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:13.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:12 smithi146 ceph-mon[33158]: osdmap e388: 8 total, 8 up, 8 in 2024-09-11T08:09:13.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:13.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:12 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:14.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:13 smithi102 ceph-mon[41371]: pgmap v569: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:09:14.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:13 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:09:14.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:13 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:09:14.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:13 smithi102 ceph-mon[41371]: osdmap e389: 8 total, 8 up, 8 in 2024-09-11T08:09:14.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:09:14.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:09:14.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:13 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:09:14.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:13 smithi102 ceph-mon[29849]: pgmap v569: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:09:14.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:13 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:09:14.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:13 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:09:14.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:13 smithi102 ceph-mon[29849]: osdmap e389: 8 total, 8 up, 8 in 2024-09-11T08:09:14.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:09:14.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:09:14.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:13 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:09:14.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:13 smithi146 ceph-mon[33158]: pgmap v569: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:09:14.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:13 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:09:14.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:13 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:09:14.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:13 smithi146 ceph-mon[33158]: osdmap e389: 8 total, 8 up, 8 in 2024-09-11T08:09:14.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:09:14.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:09:14.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:13 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:09:14.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:09:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:09:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:09:15.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:14 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:09:15.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:14 smithi102 ceph-mon[29849]: osdmap e390: 8 total, 8 up, 8 in 2024-09-11T08:09:15.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:14 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:15.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:14 smithi102 ceph-mon[29849]: pgmap v572: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:09:15.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:14 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:09:15.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:14 smithi102 ceph-mon[41371]: osdmap e390: 8 total, 8 up, 8 in 2024-09-11T08:09:15.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:14 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:15.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:14 smithi102 ceph-mon[41371]: pgmap v572: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:09:15.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:14 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:09:15.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:14 smithi146 ceph-mon[33158]: osdmap e390: 8 total, 8 up, 8 in 2024-09-11T08:09:15.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:14 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:15.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:14 smithi146 ceph-mon[33158]: pgmap v572: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:09:16.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:15 smithi102 ceph-mon[41371]: osdmap e391: 8 total, 8 up, 8 in 2024-09-11T08:09:16.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:15 smithi102 ceph-mon[29849]: osdmap e391: 8 total, 8 up, 8 in 2024-09-11T08:09:16.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:15 smithi146 ceph-mon[33158]: osdmap e391: 8 total, 8 up, 8 in 2024-09-11T08:09:17.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:16 smithi102 ceph-mon[29849]: osdmap e392: 8 total, 8 up, 8 in 2024-09-11T08:09:17.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:16 smithi102 ceph-mon[29849]: pgmap v575: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-09-11T08:09:17.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:16 smithi102 ceph-mon[41371]: osdmap e392: 8 total, 8 up, 8 in 2024-09-11T08:09:17.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:16 smithi102 ceph-mon[41371]: pgmap v575: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-09-11T08:09:17.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:16 smithi146 ceph-mon[33158]: osdmap e392: 8 total, 8 up, 8 in 2024-09-11T08:09:17.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:16 smithi146 ceph-mon[33158]: pgmap v575: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-09-11T08:09:18.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:17 smithi102 ceph-mon[29849]: osdmap e393: 8 total, 8 up, 8 in 2024-09-11T08:09:18.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:17 smithi102 ceph-mon[41371]: osdmap e393: 8 total, 8 up, 8 in 2024-09-11T08:09:18.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:17 smithi146 ceph-mon[33158]: osdmap e393: 8 total, 8 up, 8 in 2024-09-11T08:09:19.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:19 smithi102 ceph-mon[29849]: pgmap v577: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-09-11T08:09:19.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:19 smithi102 ceph-mon[41371]: pgmap v577: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-09-11T08:09:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:19 smithi146 ceph-mon[33158]: pgmap v577: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-09-11T08:09:20.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:20 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:20.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:20 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:20.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:09:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:09:19] "GET /metrics HTTP/1.1" 200 228546 "" "Prometheus/2.43.0" 2024-09-11T08:09:20.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:20 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:20.184 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:09:19 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:09:21.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:21 smithi102 ceph-mon[29849]: pgmap v578: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-11T08:09:21.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:21 smithi102 ceph-mon[41371]: pgmap v578: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-11T08:09:21.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:21 smithi146 ceph-mon[33158]: pgmap v578: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-11T08:09:22.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:22 smithi102 ceph-mon[29849]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:09:22.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:22 smithi102 ceph-mon[29849]: osdmap e394: 8 total, 8 up, 8 in 2024-09-11T08:09:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:22 smithi102 ceph-mon[41371]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:09:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:22 smithi102 ceph-mon[41371]: osdmap e394: 8 total, 8 up, 8 in 2024-09-11T08:09:22.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:22 smithi146 ceph-mon[33158]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:09:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:22 smithi146 ceph-mon[33158]: osdmap e394: 8 total, 8 up, 8 in 2024-09-11T08:09:23.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:23 smithi102 ceph-mon[29849]: pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 315 B/s wr, 3 op/s 2024-09-11T08:09:23.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:23 smithi102 ceph-mon[41371]: pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 315 B/s wr, 3 op/s 2024-09-11T08:09:23.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:23 smithi146 ceph-mon[33158]: pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 315 B/s wr, 3 op/s 2024-09-11T08:09:24.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:09:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:09:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:09:25.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:25 smithi102 ceph-mon[29849]: pgmap v581: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:09:25.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:25 smithi102 ceph-mon[41371]: pgmap v581: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:09:25.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:25 smithi146 ceph-mon[33158]: pgmap v581: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:09:27.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:27 smithi102 ceph-mon[29849]: pgmap v582: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 216 B/s wr, 2 op/s 2024-09-11T08:09:27.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:27.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:27 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:27.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-65"}]: dispatch 2024-09-11T08:09:27.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:27 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-65"}]: dispatch 2024-09-11T08:09:27.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:27 smithi102 ceph-mon[41371]: pgmap v582: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 216 B/s wr, 2 op/s 2024-09-11T08:09:27.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:27.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:27 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:27.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-65"}]: dispatch 2024-09-11T08:09:27.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:27 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-65"}]: dispatch 2024-09-11T08:09:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:27 smithi146 ceph-mon[33158]: pgmap v582: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 216 B/s wr, 2 op/s 2024-09-11T08:09:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:27 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-65"}]: dispatch 2024-09-11T08:09:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:27 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-65"}]: dispatch 2024-09-11T08:09:28.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:28 smithi102 ceph-mon[29849]: osdmap e395: 8 total, 8 up, 8 in 2024-09-11T08:09:28.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:28 smithi102 ceph-mon[41371]: osdmap e395: 8 total, 8 up, 8 in 2024-09-11T08:09:28.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:28 smithi146 ceph-mon[33158]: osdmap e395: 8 total, 8 up, 8 in 2024-09-11T08:09:29.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:29 smithi102 ceph-mon[41371]: pgmap v584: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:09:29.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:29 smithi102 ceph-mon[41371]: osdmap e396: 8 total, 8 up, 8 in 2024-09-11T08:09:29.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:29.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:29 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:29.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:29 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:29.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:29 smithi102 ceph-mon[29849]: pgmap v584: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:09:29.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:29 smithi102 ceph-mon[29849]: osdmap e396: 8 total, 8 up, 8 in 2024-09-11T08:09:29.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:29.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:29 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:29.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:29 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:29.933 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:09:29 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:09:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:29 smithi146 ceph-mon[33158]: pgmap v584: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:09:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:29 smithi146 ceph-mon[33158]: osdmap e396: 8 total, 8 up, 8 in 2024-09-11T08:09:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:29 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:29 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:30.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:09:29 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:09:29] "GET /metrics HTTP/1.1" 200 228569 "" "Prometheus/2.43.0" 2024-09-11T08:09:30.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:30 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:09:30.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:30 smithi102 ceph-mon[41371]: osdmap e397: 8 total, 8 up, 8 in 2024-09-11T08:09:30.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:30 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:09:30.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:30 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:30.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:30 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:09:30.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:30 smithi102 ceph-mon[29849]: osdmap e397: 8 total, 8 up, 8 in 2024-09-11T08:09:30.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:30 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:09:30.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:30 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:30.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:30 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:09:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:30 smithi146 ceph-mon[33158]: osdmap e397: 8 total, 8 up, 8 in 2024-09-11T08:09:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:30 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:09:30.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:30 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:31.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:31 smithi102 ceph-mon[29849]: pgmap v587: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:09:31.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:31 smithi102 ceph-mon[29849]: osdmap e398: 8 total, 8 up, 8 in 2024-09-11T08:09:31.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:31 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:31.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:31 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:31.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:31 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-67"}]: dispatch 2024-09-11T08:09:31.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:31 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-67"}]: dispatch 2024-09-11T08:09:31.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:31 smithi102 ceph-mon[41371]: pgmap v587: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:09:31.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:31 smithi102 ceph-mon[41371]: osdmap e398: 8 total, 8 up, 8 in 2024-09-11T08:09:31.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:31 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:31.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:31 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:31.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:31 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-67"}]: dispatch 2024-09-11T08:09:31.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:31 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-67"}]: dispatch 2024-09-11T08:09:31.936 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:31 smithi146 ceph-mon[33158]: pgmap v587: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:09:31.936 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:31 smithi146 ceph-mon[33158]: osdmap e398: 8 total, 8 up, 8 in 2024-09-11T08:09:31.936 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:31 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:31.936 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:31 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:31.936 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:31 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-67"}]: dispatch 2024-09-11T08:09:31.936 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:31 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-67"}]: dispatch 2024-09-11T08:09:32.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:32 smithi102 ceph-mon[29849]: osdmap e399: 8 total, 8 up, 8 in 2024-09-11T08:09:32.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:32 smithi102 ceph-mon[41371]: osdmap e399: 8 total, 8 up, 8 in 2024-09-11T08:09:32.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:32 smithi146 ceph-mon[33158]: osdmap e399: 8 total, 8 up, 8 in 2024-09-11T08:09:33.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:33 smithi102 ceph-mon[29849]: pgmap v590: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:09:33.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:33 smithi102 ceph-mon[29849]: osdmap e400: 8 total, 8 up, 8 in 2024-09-11T08:09:33.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:33.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:33 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:33.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:33 smithi102 ceph-mon[41371]: pgmap v590: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:09:33.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:33 smithi102 ceph-mon[41371]: osdmap e400: 8 total, 8 up, 8 in 2024-09-11T08:09:33.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:33.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:33 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:33.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:33 smithi146 ceph-mon[33158]: pgmap v590: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:09:33.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:33 smithi146 ceph-mon[33158]: osdmap e400: 8 total, 8 up, 8 in 2024-09-11T08:09:33.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:33.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:33 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:34.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:34 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:09:34.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:34 smithi102 ceph-mon[29849]: osdmap e401: 8 total, 8 up, 8 in 2024-09-11T08:09:34.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:34 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:09:34.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:34 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:34.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:34 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:34.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:34 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-69"}]: dispatch 2024-09-11T08:09:34.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:34 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-69"}]: dispatch 2024-09-11T08:09:34.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:34 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:09:34.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:34 smithi102 ceph-mon[41371]: osdmap e401: 8 total, 8 up, 8 in 2024-09-11T08:09:34.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:34 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:09:34.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:34 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:34.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:34 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:34.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:34 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-69"}]: dispatch 2024-09-11T08:09:34.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:34 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-69"}]: dispatch 2024-09-11T08:09:34.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:09:34 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:09:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:09:34.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:34 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:09:34.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:34 smithi146 ceph-mon[33158]: osdmap e401: 8 total, 8 up, 8 in 2024-09-11T08:09:34.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:34 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:09:34.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:34 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:34.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:34 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:34.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:34 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-69"}]: dispatch 2024-09-11T08:09:34.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:34 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-69"}]: dispatch 2024-09-11T08:09:35.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:35 smithi102 ceph-mon[29849]: pgmap v593: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:09:35.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:35 smithi102 ceph-mon[29849]: osdmap e402: 8 total, 8 up, 8 in 2024-09-11T08:09:35.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:35 smithi102 ceph-mon[41371]: pgmap v593: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:09:35.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:35 smithi102 ceph-mon[41371]: osdmap e402: 8 total, 8 up, 8 in 2024-09-11T08:09:35.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:35 smithi146 ceph-mon[33158]: pgmap v593: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:09:35.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:35 smithi146 ceph-mon[33158]: osdmap e402: 8 total, 8 up, 8 in 2024-09-11T08:09:36.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:36 smithi102 ceph-mon[41371]: osdmap e403: 8 total, 8 up, 8 in 2024-09-11T08:09:36.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:36 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:36.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:36 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:36.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:36 smithi102 ceph-mon[29849]: osdmap e403: 8 total, 8 up, 8 in 2024-09-11T08:09:36.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:36 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:36.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:36 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:36.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:36 smithi146 ceph-mon[33158]: osdmap e403: 8 total, 8 up, 8 in 2024-09-11T08:09:36.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:36 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:36.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:36 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:09:37.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:37 smithi102 ceph-mon[41371]: pgmap v596: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:09:37.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:37 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:09:37.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:37 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:09:37.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:37 smithi102 ceph-mon[41371]: osdmap e404: 8 total, 8 up, 8 in 2024-09-11T08:09:37.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:09:37.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:09:37.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:37 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:09:37.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:37 smithi102 ceph-mon[29849]: pgmap v596: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:09:37.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:37 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:09:37.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:37 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:09:37.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:37 smithi102 ceph-mon[29849]: osdmap e404: 8 total, 8 up, 8 in 2024-09-11T08:09:37.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:09:37.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:09:37.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:37 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:09:37.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:37 smithi146 ceph-mon[33158]: pgmap v596: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:09:37.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:37 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:09:37.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:37 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:09:37.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:37 smithi146 ceph-mon[33158]: osdmap e404: 8 total, 8 up, 8 in 2024-09-11T08:09:37.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:09:37.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:09:37.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:37 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:09:38.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:38 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:09:38.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:38 smithi102 ceph-mon[41371]: osdmap e405: 8 total, 8 up, 8 in 2024-09-11T08:09:38.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:38 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:09:38.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:38 smithi102 ceph-mon[29849]: osdmap e405: 8 total, 8 up, 8 in 2024-09-11T08:09:38.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:38 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:09:38.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:38 smithi146 ceph-mon[33158]: osdmap e405: 8 total, 8 up, 8 in 2024-09-11T08:09:39.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:39 smithi102 ceph-mon[29849]: pgmap v599: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:09:39.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:39 smithi102 ceph-mon[29849]: osdmap e406: 8 total, 8 up, 8 in 2024-09-11T08:09:39.896 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:09:39 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:09:39] "GET /metrics HTTP/1.1" 200 228569 "" "Prometheus/2.43.0" 2024-09-11T08:09:39.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:39 smithi102 ceph-mon[41371]: pgmap v599: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:09:39.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:39 smithi102 ceph-mon[41371]: osdmap e406: 8 total, 8 up, 8 in 2024-09-11T08:09:39.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:39 smithi146 ceph-mon[33158]: pgmap v599: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:09:39.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:39 smithi146 ceph-mon[33158]: osdmap e406: 8 total, 8 up, 8 in 2024-09-11T08:09:39.936 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:09:39 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:09:40.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:40 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:40.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:40 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:40.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:40 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:41.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:41 smithi102 ceph-mon[29849]: pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-09-11T08:09:41.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:41 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:41.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:41 smithi102 ceph-mon[41371]: pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-09-11T08:09:41.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:41 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:41.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:41 smithi146 ceph-mon[33158]: pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-09-11T08:09:41.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:41 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:09:42.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:42 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:42.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:42 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:09:42.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:42 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:09:42.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:42 smithi102 ceph-mon[29849]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:09:42.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:42 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:42.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:42 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:09:42.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:42 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:09:42.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:42 smithi102 ceph-mon[41371]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:09:42.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:42 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:09:42.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:42 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:09:42.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:42 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:09:42.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:42 smithi146 ceph-mon[33158]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:09:43.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:43 smithi146 ceph-mon[33158]: pgmap v602: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-11T08:09:44.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:43 smithi102 ceph-mon[41371]: pgmap v602: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-11T08:09:44.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:43 smithi102 ceph-mon[29849]: pgmap v602: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-11T08:09:44.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:44 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:44.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:09:44 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:09:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:09:45.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:44 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:45.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:44 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:09:45.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:45 smithi146 ceph-mon[33158]: pgmap v603: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 780 B/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:09:46.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:45 smithi102 ceph-mon[41371]: pgmap v603: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 780 B/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:09:46.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:45 smithi102 ceph-mon[29849]: pgmap v603: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 780 B/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:09:47.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:47 smithi146 ceph-mon[33158]: pgmap v604: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-11T08:09:48.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:47 smithi102 ceph-mon[41371]: pgmap v604: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-11T08:09:48.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:47 smithi102 ceph-mon[29849]: pgmap v604: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-11T08:09:49.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:49 smithi146 ceph-mon[33158]: pgmap v605: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-11T08:09:49.934 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:09:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:09:50.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:49 smithi102 ceph-mon[29849]: pgmap v605: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-11T08:09:50.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:49 smithi102 ceph-mon[41371]: pgmap v605: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-11T08:09:50.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:09:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:09:49] "GET /metrics HTTP/1.1" 200 228652 "" "Prometheus/2.43.0" 2024-09-11T08:09:51.046 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:50 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:51.046 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:50 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:51.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:50 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:09:52.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:51 smithi102 ceph-mon[29849]: pgmap v606: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-09-11T08:09:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:51 smithi102 ceph-mon[41371]: pgmap v606: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-09-11T08:09:52.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:51 smithi146 ceph-mon[33158]: pgmap v606: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-09-11T08:09:54.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:53 smithi102 ceph-mon[29849]: pgmap v607: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:09:54.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:53 smithi102 ceph-mon[41371]: pgmap v607: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:09:54.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:53 smithi146 ceph-mon[33158]: pgmap v607: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:09:54.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:09:54 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:09:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:09:56.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:55 smithi102 ceph-mon[29849]: pgmap v608: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:09:56.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:55 smithi102 ceph-mon[41371]: pgmap v608: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:09:56.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:55 smithi146 ceph-mon[33158]: pgmap v608: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:09:58.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:57 smithi102 ceph-mon[29849]: pgmap v609: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:09:58.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:57 smithi102 ceph-mon[41371]: pgmap v609: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:09:58.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:57 smithi146 ceph-mon[33158]: pgmap v609: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:09:59.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:58 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:59.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:58 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:59.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:58 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-71"}]: dispatch 2024-09-11T08:09:59.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:58 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-71"}]: dispatch 2024-09-11T08:09:59.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:58 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:59.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:58 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:59.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:58 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-71"}]: dispatch 2024-09-11T08:09:59.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:58 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-71"}]: dispatch 2024-09-11T08:09:59.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:58 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:59.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:58 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:09:59.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:58 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-71"}]: dispatch 2024-09-11T08:09:59.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:58 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-71"}]: dispatch 2024-09-11T08:10:00.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:09:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:09:59] "GET /metrics HTTP/1.1" 200 226146 "" "Prometheus/2.43.0" 2024-09-11T08:10:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:59 smithi102 ceph-mon[29849]: pgmap v610: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:10:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:59 smithi102 ceph-mon[29849]: osdmap e407: 8 total, 8 up, 8 in 2024-09-11T08:10:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:09:59 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:00.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:59 smithi102 ceph-mon[41371]: pgmap v610: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:10:00.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:59 smithi102 ceph-mon[41371]: osdmap e407: 8 total, 8 up, 8 in 2024-09-11T08:10:00.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:09:59 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:00.183 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:09:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:10:00.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:59 smithi146 ceph-mon[33158]: pgmap v610: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:10:00.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:59 smithi146 ceph-mon[33158]: osdmap e407: 8 total, 8 up, 8 in 2024-09-11T08:10:00.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:09:59 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:01.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:00 smithi102 ceph-mon[29849]: osdmap e408: 8 total, 8 up, 8 in 2024-09-11T08:10:01.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:00 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:01.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:00 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:01.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:00 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:01.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:00 smithi102 ceph-mon[29849]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-11T08:10:01.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:00 smithi102 ceph-mon[29849]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-11T08:10:01.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:00 smithi102 ceph-mon[29849]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-11T08:10:01.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:00 smithi102 ceph-mon[29849]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-11T08:10:01.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:00 smithi102 ceph-mon[41371]: osdmap e408: 8 total, 8 up, 8 in 2024-09-11T08:10:01.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:00 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:01.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:00 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:01.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:00 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:01.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:00 smithi102 ceph-mon[41371]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-11T08:10:01.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:00 smithi102 ceph-mon[41371]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-11T08:10:01.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:00 smithi102 ceph-mon[41371]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-11T08:10:01.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:00 smithi102 ceph-mon[41371]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-11T08:10:01.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:00 smithi146 ceph-mon[33158]: osdmap e408: 8 total, 8 up, 8 in 2024-09-11T08:10:01.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:00 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:01.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:00 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:01.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:00 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:01.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:00 smithi146 ceph-mon[33158]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-11T08:10:01.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:00 smithi146 ceph-mon[33158]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-11T08:10:01.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:00 smithi146 ceph-mon[33158]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-11T08:10:01.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:00 smithi146 ceph-mon[33158]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-11T08:10:02.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:01 smithi102 ceph-mon[29849]: pgmap v613: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:10:02.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:01 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:10:02.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:01 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:10:02.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:01 smithi102 ceph-mon[29849]: osdmap e409: 8 total, 8 up, 8 in 2024-09-11T08:10:02.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:01 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:10:02.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:01 smithi102 ceph-mon[41371]: pgmap v613: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:10:02.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:01 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:10:02.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:01 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:10:02.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:01 smithi102 ceph-mon[41371]: osdmap e409: 8 total, 8 up, 8 in 2024-09-11T08:10:02.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:01 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:10:02.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:01 smithi146 ceph-mon[33158]: pgmap v613: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:10:02.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:01 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:10:02.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:01 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:10:02.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:01 smithi146 ceph-mon[33158]: osdmap e409: 8 total, 8 up, 8 in 2024-09-11T08:10:02.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:01 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:10:03.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:02 smithi102 ceph-mon[29849]: osdmap e410: 8 total, 8 up, 8 in 2024-09-11T08:10:03.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:02 smithi102 ceph-mon[41371]: osdmap e410: 8 total, 8 up, 8 in 2024-09-11T08:10:03.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:02 smithi146 ceph-mon[33158]: osdmap e410: 8 total, 8 up, 8 in 2024-09-11T08:10:04.085 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:03 smithi146 ceph-mon[33158]: pgmap v616: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:10:04.085 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:03 smithi146 ceph-mon[33158]: osdmap e411: 8 total, 8 up, 8 in 2024-09-11T08:10:04.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:03 smithi102 ceph-mon[29849]: pgmap v616: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:10:04.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:03 smithi102 ceph-mon[29849]: osdmap e411: 8 total, 8 up, 8 in 2024-09-11T08:10:04.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:03 smithi102 ceph-mon[41371]: pgmap v616: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:10:04.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:03 smithi102 ceph-mon[41371]: osdmap e411: 8 total, 8 up, 8 in 2024-09-11T08:10:04.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:10:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:10:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:10:06.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:05 smithi102 ceph-mon[41371]: pgmap v618: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:10:06.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:05 smithi102 ceph-mon[29849]: pgmap v618: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:10:06.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:05 smithi146 ceph-mon[33158]: pgmap v618: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:10:08.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:07 smithi102 ceph-mon[41371]: pgmap v619: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-11T08:10:08.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:07 smithi102 ceph-mon[41371]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:10:08.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:07 smithi102 ceph-mon[29849]: pgmap v619: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-11T08:10:08.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:07 smithi102 ceph-mon[29849]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:10:08.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:07 smithi146 ceph-mon[33158]: pgmap v619: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-11T08:10:08.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:07 smithi146 ceph-mon[33158]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:10:10.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:09 smithi102 ceph-mon[41371]: pgmap v620: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 923 B/s wr, 3 op/s 2024-09-11T08:10:10.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:09 smithi102 ceph-mon[29849]: pgmap v620: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 923 B/s wr, 3 op/s 2024-09-11T08:10:10.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:10:09 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:10:09] "GET /metrics HTTP/1.1" 200 226146 "" "Prometheus/2.43.0" 2024-09-11T08:10:10.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:09 smithi146 ceph-mon[33158]: pgmap v620: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 923 B/s wr, 3 op/s 2024-09-11T08:10:10.184 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:10:09 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:10:11.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:10 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:11.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:10 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:11.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:10 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:12.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:11 smithi102 ceph-mon[41371]: pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 818 B/s wr, 3 op/s 2024-09-11T08:10:12.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:11 smithi102 ceph-mon[29849]: pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 818 B/s wr, 3 op/s 2024-09-11T08:10:12.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:11 smithi146 ceph-mon[33158]: pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 818 B/s wr, 3 op/s 2024-09-11T08:10:14.149 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:13 smithi102 ceph-mon[29849]: pgmap v622: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 716 B/s wr, 3 op/s 2024-09-11T08:10:14.149 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:14.149 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:13 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:14.149 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-73"}]: dispatch 2024-09-11T08:10:14.149 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:13 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-73"}]: dispatch 2024-09-11T08:10:14.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:13 smithi102 ceph-mon[41371]: pgmap v622: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 716 B/s wr, 3 op/s 2024-09-11T08:10:14.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:14.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:13 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:14.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-73"}]: dispatch 2024-09-11T08:10:14.150 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:13 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-73"}]: dispatch 2024-09-11T08:10:14.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:13 smithi146 ceph-mon[33158]: pgmap v622: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 716 B/s wr, 3 op/s 2024-09-11T08:10:14.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:14.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:13 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:14.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-73"}]: dispatch 2024-09-11T08:10:14.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:13 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-73"}]: dispatch 2024-09-11T08:10:14.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:14 smithi146 ceph-mon[33158]: osdmap e412: 8 total, 8 up, 8 in 2024-09-11T08:10:14.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:14 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:14.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:10:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:10:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:10:15.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:14 smithi102 ceph-mon[29849]: osdmap e412: 8 total, 8 up, 8 in 2024-09-11T08:10:15.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:14 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:15.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:14 smithi102 ceph-mon[41371]: osdmap e412: 8 total, 8 up, 8 in 2024-09-11T08:10:15.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:14 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:16.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:15 smithi102 ceph-mon[29849]: pgmap v624: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 307 B/s wr, 1 op/s 2024-09-11T08:10:16.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:15 smithi102 ceph-mon[29849]: osdmap e413: 8 total, 8 up, 8 in 2024-09-11T08:10:16.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:15 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:16.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:15 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:16.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:15 smithi102 ceph-mon[41371]: pgmap v624: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 307 B/s wr, 1 op/s 2024-09-11T08:10:16.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:15 smithi102 ceph-mon[41371]: osdmap e413: 8 total, 8 up, 8 in 2024-09-11T08:10:16.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:15 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:16.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:15 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:16.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:15 smithi146 ceph-mon[33158]: pgmap v624: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 307 B/s wr, 1 op/s 2024-09-11T08:10:16.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:15 smithi146 ceph-mon[33158]: osdmap e413: 8 total, 8 up, 8 in 2024-09-11T08:10:16.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:15 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:16.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:15 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:17.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:16 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:10:17.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:16 smithi102 ceph-mon[29849]: osdmap e414: 8 total, 8 up, 8 in 2024-09-11T08:10:17.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:10:17.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:16 smithi102 ceph-mon[29849]: osdmap e415: 8 total, 8 up, 8 in 2024-09-11T08:10:17.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:16 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:10:17.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:16 smithi102 ceph-mon[41371]: osdmap e414: 8 total, 8 up, 8 in 2024-09-11T08:10:17.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:10:17.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:16 smithi102 ceph-mon[41371]: osdmap e415: 8 total, 8 up, 8 in 2024-09-11T08:10:17.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:16 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:10:17.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:16 smithi146 ceph-mon[33158]: osdmap e414: 8 total, 8 up, 8 in 2024-09-11T08:10:17.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:10:17.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:16 smithi146 ceph-mon[33158]: osdmap e415: 8 total, 8 up, 8 in 2024-09-11T08:10:18.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:17 smithi102 ceph-mon[41371]: pgmap v627: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:10:18.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:17 smithi102 ceph-mon[29849]: pgmap v627: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:10:18.146 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 08:10:17 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-0[45637]: 2024-09-11T08:10:17.835+0000 7f4e06d05640 -1 snap_mapper.add_oid found existing snaps mapped on 64:49aaf51d:test-rados-api-smithi102-80667-76::foo:28, removing 2024-09-11T08:10:18.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:17 smithi146 ceph-mon[33158]: pgmap v627: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:10:18.466 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:10:17 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:10:17.835+0000 7fc68aba1640 -1 snap_mapper.add_oid found existing snaps mapped on 64:49aaf51d:test-rados-api-smithi102-80667-76::foo:28, removing 2024-09-11T08:10:18.467 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:10:17 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:10:17.835+0000 7fccff79e640 -1 snap_mapper.add_oid found existing snaps mapped on 64:49aaf51d:test-rados-api-smithi102-80667-76::foo:28, removing 2024-09-11T08:10:19.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:18 smithi102 ceph-mon[29849]: osdmap e416: 8 total, 8 up, 8 in 2024-09-11T08:10:19.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:19.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:18 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:19.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-75"}]: dispatch 2024-09-11T08:10:19.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:18 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-75"}]: dispatch 2024-09-11T08:10:19.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:18 smithi102 ceph-mon[41371]: osdmap e416: 8 total, 8 up, 8 in 2024-09-11T08:10:19.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:19.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:18 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:19.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-75"}]: dispatch 2024-09-11T08:10:19.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:18 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-75"}]: dispatch 2024-09-11T08:10:19.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:18 smithi146 ceph-mon[33158]: osdmap e416: 8 total, 8 up, 8 in 2024-09-11T08:10:19.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:19.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:18 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:19.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-75"}]: dispatch 2024-09-11T08:10:19.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:18 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-75"}]: dispatch 2024-09-11T08:10:20.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:19 smithi102 ceph-mon[29849]: pgmap v630: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:20.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:19 smithi102 ceph-mon[29849]: osdmap e417: 8 total, 8 up, 8 in 2024-09-11T08:10:20.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:19 smithi102 ceph-mon[41371]: pgmap v630: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:20.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:19 smithi102 ceph-mon[41371]: osdmap e417: 8 total, 8 up, 8 in 2024-09-11T08:10:20.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:10:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:10:19] "GET /metrics HTTP/1.1" 200 226141 "" "Prometheus/2.43.0" 2024-09-11T08:10:20.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:19 smithi146 ceph-mon[33158]: pgmap v630: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:10:20.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:19 smithi146 ceph-mon[33158]: osdmap e417: 8 total, 8 up, 8 in 2024-09-11T08:10:20.184 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:10:19 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:10:21.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:20 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:21.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:20 smithi102 ceph-mon[29849]: osdmap e418: 8 total, 8 up, 8 in 2024-09-11T08:10:21.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:21.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:20 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:21.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:20 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:21.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:20 smithi102 ceph-mon[41371]: osdmap e418: 8 total, 8 up, 8 in 2024-09-11T08:10:21.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:21.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:20 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:21.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:20 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:21.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:20 smithi146 ceph-mon[33158]: osdmap e418: 8 total, 8 up, 8 in 2024-09-11T08:10:21.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:21.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:20 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:22.145 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 08:10:21 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-0[45637]: 2024-09-11T08:10:21.857+0000 7f4e024fc640 -1 snap_mapper.add_oid found existing snaps mapped on 64:c51e20ef:test-rados-api-smithi102-80667-78::foo:29, removing 2024-09-11T08:10:22.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:21 smithi102 ceph-mon[41371]: pgmap v633: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-09-11T08:10:22.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:21 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:10:22.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:21 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:10:22.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:21 smithi102 ceph-mon[41371]: osdmap e419: 8 total, 8 up, 8 in 2024-09-11T08:10:22.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:21 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:10:22.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:21 smithi102 ceph-mon[29849]: pgmap v633: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-09-11T08:10:22.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:21 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:10:22.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:21 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:10:22.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:21 smithi102 ceph-mon[29849]: osdmap e419: 8 total, 8 up, 8 in 2024-09-11T08:10:22.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:21 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:10:22.147 INFO:journalctl@ceph.osd.3.smithi102.stdout:Sep 11 08:10:21 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-3[64421]: 2024-09-11T08:10:21.857+0000 7f39eeb9c640 -1 snap_mapper.add_oid found existing snaps mapped on 64:c51e20ef:test-rados-api-smithi102-80667-78::foo:29, removing 2024-09-11T08:10:22.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:21 smithi146 ceph-mon[33158]: pgmap v633: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-09-11T08:10:22.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:21 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:10:22.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:21 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:10:22.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:21 smithi146 ceph-mon[33158]: osdmap e419: 8 total, 8 up, 8 in 2024-09-11T08:10:22.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:21 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:10:22.184 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:10:21 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:10:21.858+0000 7fccfaf95640 -1 snap_mapper.add_oid found existing snaps mapped on 64:c51e20ef:test-rados-api-smithi102-80667-78::foo:29, removing 2024-09-11T08:10:23.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:22 smithi102 ceph-mon[41371]: osdmap e420: 8 total, 8 up, 8 in 2024-09-11T08:10:23.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:23.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:22 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:23.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-77"}]: dispatch 2024-09-11T08:10:23.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:22 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-77"}]: dispatch 2024-09-11T08:10:23.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:22 smithi102 ceph-mon[29849]: osdmap e420: 8 total, 8 up, 8 in 2024-09-11T08:10:23.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:23.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:22 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:23.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-77"}]: dispatch 2024-09-11T08:10:23.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:22 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-77"}]: dispatch 2024-09-11T08:10:23.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:22 smithi146 ceph-mon[33158]: osdmap e420: 8 total, 8 up, 8 in 2024-09-11T08:10:23.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:23.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:22 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:23.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-77"}]: dispatch 2024-09-11T08:10:23.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:22 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-77"}]: dispatch 2024-09-11T08:10:24.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:23 smithi102 ceph-mon[41371]: pgmap v636: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-09-11T08:10:24.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:23 smithi102 ceph-mon[41371]: osdmap e421: 8 total, 8 up, 8 in 2024-09-11T08:10:24.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:23 smithi102 ceph-mon[29849]: pgmap v636: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-09-11T08:10:24.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:23 smithi102 ceph-mon[29849]: osdmap e421: 8 total, 8 up, 8 in 2024-09-11T08:10:24.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:23 smithi146 ceph-mon[33158]: pgmap v636: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-09-11T08:10:24.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:23 smithi146 ceph-mon[33158]: osdmap e421: 8 total, 8 up, 8 in 2024-09-11T08:10:24.861 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:10:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:10:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:10:25.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:24 smithi102 ceph-mon[41371]: osdmap e422: 8 total, 8 up, 8 in 2024-09-11T08:10:25.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:25.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:24 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:25.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:24 smithi102 ceph-mon[29849]: osdmap e422: 8 total, 8 up, 8 in 2024-09-11T08:10:25.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:25.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:24 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:25.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:24 smithi146 ceph-mon[33158]: osdmap e422: 8 total, 8 up, 8 in 2024-09-11T08:10:25.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:25.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:24 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:26.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:25 smithi102 ceph-mon[41371]: pgmap v639: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:10:26.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:25 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:10:26.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:25 smithi102 ceph-mon[41371]: osdmap e423: 8 total, 8 up, 8 in 2024-09-11T08:10:26.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:10:26.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:25 smithi102 ceph-mon[29849]: pgmap v639: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:10:26.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:25 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:10:26.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:25 smithi102 ceph-mon[29849]: osdmap e423: 8 total, 8 up, 8 in 2024-09-11T08:10:26.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:10:26.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:25 smithi146 ceph-mon[33158]: pgmap v639: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:10:26.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:25 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:10:26.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:25 smithi146 ceph-mon[33158]: osdmap e423: 8 total, 8 up, 8 in 2024-09-11T08:10:26.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:10:27.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:26 smithi102 ceph-mon[41371]: osdmap e424: 8 total, 8 up, 8 in 2024-09-11T08:10:27.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:26 smithi102 ceph-mon[29849]: osdmap e424: 8 total, 8 up, 8 in 2024-09-11T08:10:27.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:26 smithi146 ceph-mon[33158]: osdmap e424: 8 total, 8 up, 8 in 2024-09-11T08:10:28.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:27 smithi102 ceph-mon[41371]: pgmap v642: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-11T08:10:28.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:27 smithi102 ceph-mon[41371]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:10:28.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:27 smithi102 ceph-mon[41371]: osdmap e425: 8 total, 8 up, 8 in 2024-09-11T08:10:28.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.13"}]: dispatch 2024-09-11T08:10:28.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:27 smithi102 ceph-mon[41371]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.13"}]: dispatch 2024-09-11T08:10:28.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:27 smithi102 ceph-mon[29849]: pgmap v642: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-11T08:10:28.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:27 smithi102 ceph-mon[29849]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:10:28.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:27 smithi102 ceph-mon[29849]: osdmap e425: 8 total, 8 up, 8 in 2024-09-11T08:10:28.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.13"}]: dispatch 2024-09-11T08:10:28.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:27 smithi102 ceph-mon[29849]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.13"}]: dispatch 2024-09-11T08:10:28.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:27 smithi146 ceph-mon[33158]: pgmap v642: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-11T08:10:28.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:27 smithi146 ceph-mon[33158]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:10:28.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:27 smithi146 ceph-mon[33158]: osdmap e425: 8 total, 8 up, 8 in 2024-09-11T08:10:28.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.13"}]: dispatch 2024-09-11T08:10:28.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:27 smithi146 ceph-mon[33158]: from='mon.? v1:172.21.15.146:0/4185739029' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.13"}]: dispatch 2024-09-11T08:10:30.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:10:29 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:10:29] "GET /metrics HTTP/1.1" 200 228648 "" "Prometheus/2.43.0" 2024-09-11T08:10:30.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:29 smithi102 ceph-mon[29849]: pgmap v644: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-11T08:10:30.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:29 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:29 smithi102 ceph-mon[41371]: pgmap v644: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-11T08:10:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:29 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:30.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:29 smithi146 ceph-mon[33158]: pgmap v644: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-11T08:10:30.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:29 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:30.184 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:10:29 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:10:31.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:30 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:31.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:30 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:31.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:30 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:32.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:31 smithi102 ceph-mon[29849]: pgmap v645: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-09-11T08:10:32.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:31 smithi102 ceph-mon[41371]: pgmap v645: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-09-11T08:10:32.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:31 smithi146 ceph-mon[33158]: pgmap v645: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-09-11T08:10:34.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:33 smithi102 ceph-mon[29849]: pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-11T08:10:34.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:33 smithi102 ceph-mon[41371]: pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-11T08:10:34.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:33 smithi146 ceph-mon[33158]: pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-11T08:10:34.896 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:10:34 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:10:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:10:35.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:34 smithi146 ceph-mon[33158]: 223.13 deep-scrub starts 2024-09-11T08:10:35.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:34 smithi146 ceph-mon[33158]: 223.13 deep-scrub ok 2024-09-11T08:10:35.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:34 smithi102 ceph-mon[41371]: 223.13 deep-scrub starts 2024-09-11T08:10:35.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:34 smithi102 ceph-mon[41371]: 223.13 deep-scrub ok 2024-09-11T08:10:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:34 smithi102 ceph-mon[29849]: 223.13 deep-scrub starts 2024-09-11T08:10:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:34 smithi102 ceph-mon[29849]: 223.13 deep-scrub ok 2024-09-11T08:10:36.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:35 smithi146 ceph-mon[33158]: pgmap v647: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 592 B/s rd, 237 B/s wr, 1 op/s 2024-09-11T08:10:36.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:35 smithi102 ceph-mon[41371]: pgmap v647: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 592 B/s rd, 237 B/s wr, 1 op/s 2024-09-11T08:10:36.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:35 smithi102 ceph-mon[29849]: pgmap v647: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 592 B/s rd, 237 B/s wr, 1 op/s 2024-09-11T08:10:38.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:37 smithi146 ceph-mon[33158]: pgmap v648: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-09-11T08:10:38.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:37 smithi102 ceph-mon[41371]: pgmap v648: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-09-11T08:10:38.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:37 smithi102 ceph-mon[29849]: pgmap v648: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-09-11T08:10:40.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:39 smithi102 ceph-mon[41371]: pgmap v649: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 880 B/s rd, 176 B/s wr, 1 op/s 2024-09-11T08:10:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:39 smithi102 ceph-mon[29849]: pgmap v649: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 880 B/s rd, 176 B/s wr, 1 op/s 2024-09-11T08:10:40.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:10:39 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:10:39] "GET /metrics HTTP/1.1" 200 228648 "" "Prometheus/2.43.0" 2024-09-11T08:10:40.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:39 smithi146 ceph-mon[33158]: pgmap v649: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 880 B/s rd, 176 B/s wr, 1 op/s 2024-09-11T08:10:40.184 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:10:39 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:10:41.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:40 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:41.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:40 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:41.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:40 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:42.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:41 smithi146 ceph-mon[33158]: pgmap v650: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-09-11T08:10:42.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:41 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:10:42.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:41 smithi102 ceph-mon[41371]: pgmap v650: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-09-11T08:10:42.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:41 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:10:42.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:41 smithi102 ceph-mon[29849]: pgmap v650: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-09-11T08:10:42.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:41 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:10:43.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:42 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:43.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:42 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:10:43.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:42 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:10:43.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:42 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:43.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:42 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:10:43.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:42 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:10:43.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:42 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:10:43.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:42 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:10:43.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:42 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:10:44.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:43 smithi146 ceph-mon[33158]: pgmap v651: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:44.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:43 smithi102 ceph-mon[41371]: pgmap v651: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:44.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:43 smithi102 ceph-mon[29849]: pgmap v651: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:44.916 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:10:44 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:10:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:10:45.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:44 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:45.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:44 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:45.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:44 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:10:46.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:45 smithi146 ceph-mon[33158]: pgmap v652: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:46.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:45 smithi102 ceph-mon[41371]: pgmap v652: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:46.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:45 smithi102 ceph-mon[29849]: pgmap v652: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:10:47.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:46 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:47.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:46 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:47.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:46 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-79"}]: dispatch 2024-09-11T08:10:47.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:46 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-79"}]: dispatch 2024-09-11T08:10:47.184 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 11 08:10:46 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-4[38466]: 2024-09-11T08:10:46.911+0000 7f87faa35640 -1 snap_mapper.add_oid found existing snaps mapped on 223:cb9123df:test-rados-api-smithi102-80667-80::foo:2, removing 2024-09-11T08:10:47.395 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 08:10:46 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-0[45637]: 2024-09-11T08:10:46.911+0000 7f4e0a50c640 -1 snap_mapper.add_oid found existing snaps mapped on 223:cb9123df:test-rados-api-smithi102-80667-80::foo:2, removing 2024-09-11T08:10:47.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:46 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:47.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:46 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:47.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:46 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-79"}]: dispatch 2024-09-11T08:10:47.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:46 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-79"}]: dispatch 2024-09-11T08:10:47.396 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 08:10:46 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-2[58066]: 2024-09-11T08:10:46.911+0000 7f33dc5df640 -1 snap_mapper.add_oid found existing snaps mapped on 223:cb9123df:test-rados-api-smithi102-80667-80::foo:2, removing 2024-09-11T08:10:47.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:47.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:46 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:47.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-79"}]: dispatch 2024-09-11T08:10:47.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:46 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-79"}]: dispatch 2024-09-11T08:10:48.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:47 smithi146 ceph-mon[33158]: pgmap v653: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-11T08:10:48.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:47 smithi102 ceph-mon[29849]: pgmap v653: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-11T08:10:48.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:47 smithi102 ceph-mon[41371]: pgmap v653: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-09-11T08:10:49.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:48 smithi102 ceph-mon[29849]: osdmap e426: 8 total, 8 up, 8 in 2024-09-11T08:10:49.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:48 smithi102 ceph-mon[41371]: osdmap e426: 8 total, 8 up, 8 in 2024-09-11T08:10:49.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:48 smithi146 ceph-mon[33158]: osdmap e426: 8 total, 8 up, 8 in 2024-09-11T08:10:50.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:10:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:10:49] "GET /metrics HTTP/1.1" 200 228606 "" "Prometheus/2.43.0" 2024-09-11T08:10:50.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:49 smithi102 ceph-mon[29849]: pgmap v655: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s 2024-09-11T08:10:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:49 smithi102 ceph-mon[29849]: osdmap e427: 8 total, 8 up, 8 in 2024-09-11T08:10:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:49 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:49 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:50.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:49 smithi102 ceph-mon[41371]: pgmap v655: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s 2024-09-11T08:10:50.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:49 smithi102 ceph-mon[41371]: osdmap e427: 8 total, 8 up, 8 in 2024-09-11T08:10:50.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:49 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:50.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:49 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:50.183 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:10:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:10:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:49 smithi146 ceph-mon[33158]: pgmap v655: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 44 KiB/s rd, 0 B/s wr, 72 op/s 2024-09-11T08:10:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:49 smithi146 ceph-mon[33158]: osdmap e427: 8 total, 8 up, 8 in 2024-09-11T08:10:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:49 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:49 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:10:51.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:50 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:51.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:50 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:10:51.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:50 smithi102 ceph-mon[41371]: osdmap e428: 8 total, 8 up, 8 in 2024-09-11T08:10:51.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:50 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:10:51.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:50 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:10:51.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:50 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:10:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:50 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:50 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:10:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:50 smithi102 ceph-mon[29849]: osdmap e428: 8 total, 8 up, 8 in 2024-09-11T08:10:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:50 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:10:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:50 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:10:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:50 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:10:51.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:50 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:10:51.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:50 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:10:51.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:50 smithi146 ceph-mon[33158]: osdmap e428: 8 total, 8 up, 8 in 2024-09-11T08:10:51.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:50 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:10:51.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:50 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:10:51.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:50 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:10:52.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:51 smithi102 ceph-mon[29849]: pgmap v658: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 74 KiB/s rd, 0 B/s wr, 120 op/s 2024-09-11T08:10:52.400 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:51 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:10:52.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:51 smithi102 ceph-mon[29849]: osdmap e429: 8 total, 8 up, 8 in 2024-09-11T08:10:52.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:51 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:52.401 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:51 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:52.401 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:51 smithi102 ceph-mon[41371]: pgmap v658: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 74 KiB/s rd, 0 B/s wr, 120 op/s 2024-09-11T08:10:52.401 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:51 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:10:52.401 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:51 smithi102 ceph-mon[41371]: osdmap e429: 8 total, 8 up, 8 in 2024-09-11T08:10:52.402 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:51 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:52.402 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:51 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:52.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:51 smithi146 ceph-mon[33158]: pgmap v658: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 74 KiB/s rd, 0 B/s wr, 120 op/s 2024-09-11T08:10:52.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:51 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:10:52.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:51 smithi146 ceph-mon[33158]: osdmap e429: 8 total, 8 up, 8 in 2024-09-11T08:10:52.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:51 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:52.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:51 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:53.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:52 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:10:53.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:52 smithi102 ceph-mon[29849]: osdmap e430: 8 total, 8 up, 8 in 2024-09-11T08:10:53.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:52 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:10:53.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:52 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:10:53.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:52 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:10:53.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:52 smithi102 ceph-mon[41371]: osdmap e430: 8 total, 8 up, 8 in 2024-09-11T08:10:53.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:52 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:10:53.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:52 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:10:53.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:52 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:10:53.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:52 smithi146 ceph-mon[33158]: osdmap e430: 8 total, 8 up, 8 in 2024-09-11T08:10:53.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:52 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:10:53.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:52 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:10:54.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:53 smithi102 ceph-mon[29849]: pgmap v661: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-11T08:10:54.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:53 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:10:54.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:53 smithi102 ceph-mon[29849]: osdmap e431: 8 total, 8 up, 8 in 2024-09-11T08:10:54.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:53 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:10:54.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:53 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:10:54.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:53 smithi102 ceph-mon[41371]: pgmap v661: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-11T08:10:54.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:53 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:10:54.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:53 smithi102 ceph-mon[41371]: osdmap e431: 8 total, 8 up, 8 in 2024-09-11T08:10:54.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:53 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:10:54.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:53 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:10:54.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:53 smithi146 ceph-mon[33158]: pgmap v661: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-11T08:10:54.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:53 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:10:54.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:53 smithi146 ceph-mon[33158]: osdmap e431: 8 total, 8 up, 8 in 2024-09-11T08:10:54.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:53 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:10:54.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:53 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:10:54.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:10:54 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:10:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:10:55.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:54 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:10:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:54 smithi102 ceph-mon[29849]: osdmap e432: 8 total, 8 up, 8 in 2024-09-11T08:10:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:54 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-11T08:10:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:54 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-11T08:10:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:54 smithi102 ceph-mon[29849]: pgmap v664: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:10:55.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:54 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:10:55.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:54 smithi102 ceph-mon[41371]: osdmap e432: 8 total, 8 up, 8 in 2024-09-11T08:10:55.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:54 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-11T08:10:55.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:54 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-11T08:10:55.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:54 smithi102 ceph-mon[41371]: pgmap v664: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:10:55.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:54 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:10:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:54 smithi146 ceph-mon[33158]: osdmap e432: 8 total, 8 up, 8 in 2024-09-11T08:10:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:54 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-11T08:10:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:54 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-11T08:10:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:54 smithi146 ceph-mon[33158]: pgmap v664: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:10:56.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:55 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-11T08:10:56.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:55 smithi102 ceph-mon[29849]: osdmap e433: 8 total, 8 up, 8 in 2024-09-11T08:10:56.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-11T08:10:56.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:55 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-11T08:10:56.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:55 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-11T08:10:56.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:55 smithi102 ceph-mon[41371]: osdmap e433: 8 total, 8 up, 8 in 2024-09-11T08:10:56.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-11T08:10:56.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:55 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-11T08:10:56.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:55 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-11T08:10:56.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:55 smithi146 ceph-mon[33158]: osdmap e433: 8 total, 8 up, 8 in 2024-09-11T08:10:56.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-11T08:10:56.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:55 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-11T08:10:57.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:57 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-11T08:10:57.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:57 smithi102 ceph-mon[29849]: osdmap e434: 8 total, 8 up, 8 in 2024-09-11T08:10:57.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:57 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:10:57.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:57 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:10:57.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:57 smithi102 ceph-mon[29849]: pgmap v667: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 9.2 KiB/s wr, 21 op/s 2024-09-11T08:10:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:57 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-11T08:10:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:57 smithi102 ceph-mon[41371]: osdmap e434: 8 total, 8 up, 8 in 2024-09-11T08:10:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:57 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:10:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:57 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:10:57.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:57 smithi102 ceph-mon[41371]: pgmap v667: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 9.2 KiB/s wr, 21 op/s 2024-09-11T08:10:57.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:56 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-11T08:10:57.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:56 smithi146 ceph-mon[33158]: osdmap e434: 8 total, 8 up, 8 in 2024-09-11T08:10:57.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:56 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:10:57.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:56 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:10:57.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:56 smithi146 ceph-mon[33158]: pgmap v667: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 9.2 KiB/s wr, 21 op/s 2024-09-11T08:10:58.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:58 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:10:58.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:58 smithi102 ceph-mon[29849]: osdmap e435: 8 total, 8 up, 8 in 2024-09-11T08:10:58.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:58 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:58.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:58 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:58.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:58 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-81"}]: dispatch 2024-09-11T08:10:58.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:58 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-81"}]: dispatch 2024-09-11T08:10:58.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:58 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:10:58.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:58 smithi102 ceph-mon[41371]: osdmap e435: 8 total, 8 up, 8 in 2024-09-11T08:10:58.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:58 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:58.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:58 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:58.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:58 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-81"}]: dispatch 2024-09-11T08:10:58.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:58 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-81"}]: dispatch 2024-09-11T08:10:58.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:58 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:10:58.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:58 smithi146 ceph-mon[33158]: osdmap e435: 8 total, 8 up, 8 in 2024-09-11T08:10:58.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:58 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:58.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:58 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:10:58.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:58 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-81"}]: dispatch 2024-09-11T08:10:58.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:58 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-81"}]: dispatch 2024-09-11T08:10:59.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:59 smithi102 ceph-mon[41371]: osdmap e436: 8 total, 8 up, 8 in 2024-09-11T08:10:59.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:10:59 smithi102 ceph-mon[41371]: pgmap v670: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 7.0 KiB/s wr, 21 op/s 2024-09-11T08:10:59.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:59 smithi102 ceph-mon[29849]: osdmap e436: 8 total, 8 up, 8 in 2024-09-11T08:10:59.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:10:59 smithi102 ceph-mon[29849]: pgmap v670: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 7.0 KiB/s wr, 21 op/s 2024-09-11T08:10:59.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:59 smithi146 ceph-mon[33158]: osdmap e436: 8 total, 8 up, 8 in 2024-09-11T08:10:59.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:10:59 smithi146 ceph-mon[33158]: pgmap v670: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 7.0 KiB/s wr, 21 op/s 2024-09-11T08:11:00.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:10:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:10:59] "GET /metrics HTTP/1.1" 200 228632 "" "Prometheus/2.43.0" 2024-09-11T08:11:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:00 smithi102 ceph-mon[29849]: osdmap e437: 8 total, 8 up, 8 in 2024-09-11T08:11:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:00 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:00 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:00 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:00 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:00.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:00 smithi102 ceph-mon[41371]: osdmap e437: 8 total, 8 up, 8 in 2024-09-11T08:11:00.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:00 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:00.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:00 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:00.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:00 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:00.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:00 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:00.184 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:10:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:11:00.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:00 smithi146 ceph-mon[33158]: osdmap e437: 8 total, 8 up, 8 in 2024-09-11T08:11:00.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:00 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:00.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:00 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:00.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:00 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:00.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:00 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:01.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:01 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:11:01.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:01 smithi102 ceph-mon[29849]: osdmap e438: 8 total, 8 up, 8 in 2024-09-11T08:11:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:01 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:11:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:01 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:01 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:01 smithi102 ceph-mon[29849]: pgmap v673: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 18 op/s 2024-09-11T08:11:01.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:01 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:11:01.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:01 smithi102 ceph-mon[41371]: osdmap e438: 8 total, 8 up, 8 in 2024-09-11T08:11:01.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:01 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:11:01.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:01 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:01.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:01 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:01.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:01 smithi102 ceph-mon[41371]: pgmap v673: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 18 op/s 2024-09-11T08:11:01.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:01 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:11:01.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:01 smithi146 ceph-mon[33158]: osdmap e438: 8 total, 8 up, 8 in 2024-09-11T08:11:01.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:01 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:11:01.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:01 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:01.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:01 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:01.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:01 smithi146 ceph-mon[33158]: pgmap v673: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 3.0 KiB/s wr, 18 op/s 2024-09-11T08:11:02.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:02 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:11:02.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:02 smithi102 ceph-mon[41371]: osdmap e439: 8 total, 8 up, 8 in 2024-09-11T08:11:02.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:02 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:02.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:02 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:02.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:02 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:11:02.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:02 smithi102 ceph-mon[41371]: osdmap e440: 8 total, 8 up, 8 in 2024-09-11T08:11:02.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:02 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:11:02.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:02 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:11:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:02 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:11:02.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:02 smithi102 ceph-mon[29849]: osdmap e439: 8 total, 8 up, 8 in 2024-09-11T08:11:02.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:02 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:02.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:02 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:02.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:02 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:11:02.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:02 smithi102 ceph-mon[29849]: osdmap e440: 8 total, 8 up, 8 in 2024-09-11T08:11:02.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:02 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:11:02.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:02 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:11:02.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:02 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:11:02.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:02 smithi146 ceph-mon[33158]: osdmap e439: 8 total, 8 up, 8 in 2024-09-11T08:11:02.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:02 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:02.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:02 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:02.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:02 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:11:02.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:02 smithi146 ceph-mon[33158]: osdmap e440: 8 total, 8 up, 8 in 2024-09-11T08:11:02.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:02 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:11:02.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:02 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:11:03.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:03 smithi102 ceph-mon[41371]: pgmap v676: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-09-11T08:11:03.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:03 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:11:03.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:03 smithi102 ceph-mon[41371]: osdmap e441: 8 total, 8 up, 8 in 2024-09-11T08:11:03.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:03 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:11:03.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:03 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:11:03.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:03 smithi102 ceph-mon[29849]: pgmap v676: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-09-11T08:11:03.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:03 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:11:03.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:03 smithi102 ceph-mon[29849]: osdmap e441: 8 total, 8 up, 8 in 2024-09-11T08:11:03.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:03 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:11:03.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:03 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:11:03.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:03 smithi146 ceph-mon[33158]: pgmap v676: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 7.2 KiB/s rd, 5.2 KiB/s wr, 19 op/s 2024-09-11T08:11:03.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:03 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:11:03.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:03 smithi146 ceph-mon[33158]: osdmap e441: 8 total, 8 up, 8 in 2024-09-11T08:11:03.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:03 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:11:03.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:03 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:11:04.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:11:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:11:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:11:05.213 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:05 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:11:05.213 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:05 smithi146 ceph-mon[33158]: osdmap e442: 8 total, 8 up, 8 in 2024-09-11T08:11:05.213 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:05 smithi146 ceph-mon[33158]: pgmap v679: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:11:05.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:05 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:11:05.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:05 smithi102 ceph-mon[41371]: osdmap e442: 8 total, 8 up, 8 in 2024-09-11T08:11:05.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:05 smithi102 ceph-mon[41371]: pgmap v679: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:11:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:05 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:11:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:05 smithi102 ceph-mon[29849]: osdmap e442: 8 total, 8 up, 8 in 2024-09-11T08:11:05.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:05 smithi102 ceph-mon[29849]: pgmap v679: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:11:06.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:06 smithi102 ceph-mon[41371]: osdmap e443: 8 total, 8 up, 8 in 2024-09-11T08:11:06.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:06 smithi102 ceph-mon[29849]: osdmap e443: 8 total, 8 up, 8 in 2024-09-11T08:11:06.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:06 smithi146 ceph-mon[33158]: osdmap e443: 8 total, 8 up, 8 in 2024-09-11T08:11:07.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:07 smithi146 ceph-mon[33158]: osdmap e444: 8 total, 8 up, 8 in 2024-09-11T08:11:07.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:07.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:07 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:07.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-83"}]: dispatch 2024-09-11T08:11:07.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:07 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-83"}]: dispatch 2024-09-11T08:11:07.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:07 smithi146 ceph-mon[33158]: pgmap v682: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-09-11T08:11:07.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:07 smithi102 ceph-mon[41371]: osdmap e444: 8 total, 8 up, 8 in 2024-09-11T08:11:07.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:07.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:07 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:07.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-83"}]: dispatch 2024-09-11T08:11:07.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:07 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-83"}]: dispatch 2024-09-11T08:11:07.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:07 smithi102 ceph-mon[41371]: pgmap v682: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-09-11T08:11:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:07 smithi102 ceph-mon[29849]: osdmap e444: 8 total, 8 up, 8 in 2024-09-11T08:11:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:07 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-83"}]: dispatch 2024-09-11T08:11:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:07 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-83"}]: dispatch 2024-09-11T08:11:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:07 smithi102 ceph-mon[29849]: pgmap v682: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-09-11T08:11:08.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:08 smithi102 ceph-mon[41371]: osdmap e445: 8 total, 8 up, 8 in 2024-09-11T08:11:08.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:08 smithi102 ceph-mon[29849]: osdmap e445: 8 total, 8 up, 8 in 2024-09-11T08:11:08.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:08 smithi146 ceph-mon[33158]: osdmap e445: 8 total, 8 up, 8 in 2024-09-11T08:11:09.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:09 smithi102 ceph-mon[41371]: osdmap e446: 8 total, 8 up, 8 in 2024-09-11T08:11:09.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:09 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:09.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:09 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:09.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:09 smithi102 ceph-mon[41371]: pgmap v685: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:11:09.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:09 smithi102 ceph-mon[29849]: osdmap e446: 8 total, 8 up, 8 in 2024-09-11T08:11:09.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:09 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:09.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:09 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:09.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:09 smithi102 ceph-mon[29849]: pgmap v685: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:11:09.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:09 smithi146 ceph-mon[33158]: osdmap e446: 8 total, 8 up, 8 in 2024-09-11T08:11:09.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:09 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:09.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:09 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:09.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:09 smithi146 ceph-mon[33158]: pgmap v685: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:11:10.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:11:09 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:11:09] "GET /metrics HTTP/1.1" 200 228632 "" "Prometheus/2.43.0" 2024-09-11T08:11:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:10 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:11:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:10 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:11:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:10 smithi102 ceph-mon[41371]: osdmap e447: 8 total, 8 up, 8 in 2024-09-11T08:11:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:11:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:10 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:10 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:10 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:11:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:10 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:11:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:10 smithi102 ceph-mon[29849]: osdmap e447: 8 total, 8 up, 8 in 2024-09-11T08:11:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:11:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:10 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:10.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:10 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:10.183 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:11:09 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:11:10.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:10 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:11:10.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:10 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:11:10.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:10 smithi146 ceph-mon[33158]: osdmap e447: 8 total, 8 up, 8 in 2024-09-11T08:11:10.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:11:10.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:10.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:10 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:10.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:10 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:11.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:11 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:11:11.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:11 smithi102 ceph-mon[29849]: osdmap e448: 8 total, 8 up, 8 in 2024-09-11T08:11:11.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:11 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:11.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:11 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:11.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:11 smithi102 ceph-mon[29849]: pgmap v688: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-11T08:11:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:11 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:11:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:11 smithi102 ceph-mon[41371]: osdmap e448: 8 total, 8 up, 8 in 2024-09-11T08:11:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:11 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:11 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:11 smithi102 ceph-mon[41371]: pgmap v688: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-11T08:11:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:11 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:11:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:11 smithi146 ceph-mon[33158]: osdmap e448: 8 total, 8 up, 8 in 2024-09-11T08:11:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:11 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:11 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:11 smithi146 ceph-mon[33158]: pgmap v688: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-11T08:11:12.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:12 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:11:12.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:12 smithi102 ceph-mon[41371]: osdmap e449: 8 total, 8 up, 8 in 2024-09-11T08:11:12.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:11:12.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:12 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:11:12.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:12 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:11:12.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:12 smithi102 ceph-mon[29849]: osdmap e449: 8 total, 8 up, 8 in 2024-09-11T08:11:12.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:11:12.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:12 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:11:12.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:12 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_tier","val": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:11:12.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:12 smithi146 ceph-mon[33158]: osdmap e449: 8 total, 8 up, 8 in 2024-09-11T08:11:12.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:11:12.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:12 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:11:13.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:13 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:11:13.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:13 smithi102 ceph-mon[29849]: osdmap e450: 8 total, 8 up, 8 in 2024-09-11T08:11:13.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:11:13.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:13 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:11:13.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:13 smithi102 ceph-mon[29849]: pgmap v691: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-09-11T08:11:13.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:13 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:11:13.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:13 smithi102 ceph-mon[41371]: osdmap e450: 8 total, 8 up, 8 in 2024-09-11T08:11:13.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:11:13.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:13 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:11:13.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:13 smithi102 ceph-mon[41371]: pgmap v691: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-09-11T08:11:13.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:13 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:11:13.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:13 smithi146 ceph-mon[33158]: osdmap e450: 8 total, 8 up, 8 in 2024-09-11T08:11:13.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:11:13.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:13 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:11:13.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:13 smithi146 ceph-mon[33158]: pgmap v691: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-09-11T08:11:14.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:14 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:11:14.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:14 smithi102 ceph-mon[29849]: osdmap e451: 8 total, 8 up, 8 in 2024-09-11T08:11:14.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:14 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:11:14.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:14 smithi102 ceph-mon[41371]: osdmap e451: 8 total, 8 up, 8 in 2024-09-11T08:11:14.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:14 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:11:14.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:14 smithi146 ceph-mon[33158]: osdmap e451: 8 total, 8 up, 8 in 2024-09-11T08:11:14.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:11:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:11:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:11:15.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:15 smithi102 ceph-mon[29849]: osdmap e452: 8 total, 8 up, 8 in 2024-09-11T08:11:15.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:15.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:15 smithi102 ceph-mon[29849]: pgmap v694: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:11:15.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.0", "id": [6, 3]}]: dispatch 2024-09-11T08:11:15.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.b", "id": [6, 3]}]: dispatch 2024-09-11T08:11:15.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:15 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.0", "id": [6, 3]}]: dispatch 2024-09-11T08:11:15.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.17", "id": [6, 1]}]: dispatch 2024-09-11T08:11:15.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:15 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.b", "id": [6, 3]}]: dispatch 2024-09-11T08:11:15.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:15 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.17", "id": [6, 1]}]: dispatch 2024-09-11T08:11:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:15 smithi102 ceph-mon[41371]: osdmap e452: 8 total, 8 up, 8 in 2024-09-11T08:11:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:15 smithi102 ceph-mon[41371]: pgmap v694: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:11:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.0", "id": [6, 3]}]: dispatch 2024-09-11T08:11:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.b", "id": [6, 3]}]: dispatch 2024-09-11T08:11:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:15 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.0", "id": [6, 3]}]: dispatch 2024-09-11T08:11:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.17", "id": [6, 1]}]: dispatch 2024-09-11T08:11:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:15 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.b", "id": [6, 3]}]: dispatch 2024-09-11T08:11:15.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:15 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.17", "id": [6, 1]}]: dispatch 2024-09-11T08:11:15.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:15 smithi146 ceph-mon[33158]: osdmap e452: 8 total, 8 up, 8 in 2024-09-11T08:11:15.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:15.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:15 smithi146 ceph-mon[33158]: pgmap v694: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:11:15.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.0", "id": [6, 3]}]: dispatch 2024-09-11T08:11:15.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.b", "id": [6, 3]}]: dispatch 2024-09-11T08:11:15.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:15 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.0", "id": [6, 3]}]: dispatch 2024-09-11T08:11:15.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.17", "id": [6, 1]}]: dispatch 2024-09-11T08:11:15.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:15 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.b", "id": [6, 3]}]: dispatch 2024-09-11T08:11:15.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:15 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.17", "id": [6, 1]}]: dispatch 2024-09-11T08:11:16.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:16 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.0", "id": [6, 3]}]': finished 2024-09-11T08:11:16.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:16 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.b", "id": [6, 3]}]': finished 2024-09-11T08:11:16.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:16 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.17", "id": [6, 1]}]': finished 2024-09-11T08:11:16.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:16 smithi102 ceph-mon[41371]: osdmap e453: 8 total, 8 up, 8 in 2024-09-11T08:11:16.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:16 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.0", "id": [6, 3]}]': finished 2024-09-11T08:11:16.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:16 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.b", "id": [6, 3]}]': finished 2024-09-11T08:11:16.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:16 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.17", "id": [6, 1]}]': finished 2024-09-11T08:11:16.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:16 smithi102 ceph-mon[29849]: osdmap e453: 8 total, 8 up, 8 in 2024-09-11T08:11:16.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:16 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.0", "id": [6, 3]}]': finished 2024-09-11T08:11:16.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:16 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.b", "id": [6, 3]}]': finished 2024-09-11T08:11:16.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:16 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "226.17", "id": [6, 1]}]': finished 2024-09-11T08:11:16.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:16 smithi146 ceph-mon[33158]: osdmap e453: 8 total, 8 up, 8 in 2024-09-11T08:11:17.127 INFO:tasks.workunit.client.0.smithi102.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (17845 ms) 2024-09-11T08:11:17.127 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-09-11T08:11:17.127 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22338 ms) 2024-09-11T08:11:17.127 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-09-11T08:11:17.127 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3008 ms) 2024-09-11T08:11:17.128 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-09-11T08:11:17.128 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:11:17.128 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3020 ms) 2024-09-11T08:11:17.128 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-09-11T08:11:17.128 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3022 ms) 2024-09-11T08:11:17.128 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-09-11T08:11:17.128 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:11:17.128 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3021 ms) 2024-09-11T08:11:17.128 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-09-11T08:11:17.128 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:11:17.128 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4029 ms) 2024-09-11T08:11:17.128 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-09-11T08:11:17.129 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:11:17.129 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (39305 ms) 2024-09-11T08:11:17.129 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-09-11T08:11:17.129 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:11:17.129 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (16600 ms) 2024-09-11T08:11:17.129 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-09-11T08:11:17.129 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:11:17.129 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4036 ms) 2024-09-11T08:11:17.129 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-09-11T08:11:17.129 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:11:17.129 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3013 ms) 2024-09-11T08:11:17.129 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-09-11T08:11:17.129 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:11:17.129 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (24123 ms) 2024-09-11T08:11:17.130 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-09-11T08:11:17.130 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:11:17.130 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (15059 ms) 2024-09-11T08:11:17.130 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-09-11T08:11:17.130 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:11:17.130 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5052 ms) 2024-09-11T08:11:17.130 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-09-11T08:11:17.130 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:11:17.130 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4028 ms) 2024-09-11T08:11:17.130 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-09-11T08:11:17.130 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:11:17.130 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: waiting for scrubs... 2024-09-11T08:11:17.130 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: done waiting 2024-09-11T08:11:17.131 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (25078 ms) 2024-09-11T08:11:17.131 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-09-11T08:11:17.131 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:11:17.131 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10080 ms) 2024-09-11T08:11:17.131 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-09-11T08:11:17.131 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:11:17.131 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9050 ms) 2024-09-11T08:11:17.131 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-09-11T08:11:17.131 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:11:17.131 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (10066 ms) 2024-09-11T08:11:17.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:17 smithi102 ceph-mon[29849]: osdmap e454: 8 total, 8 up, 8 in 2024-09-11T08:11:17.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:17.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:17 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:17.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-85"}]: dispatch 2024-09-11T08:11:17.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:17 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-85"}]: dispatch 2024-09-11T08:11:17.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:17 smithi102 ceph-mon[29849]: pgmap v697: 228 pgs: 3 peering, 225 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 6.0 KiB/s wr, 9 op/s 2024-09-11T08:11:17.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:17 smithi102 ceph-mon[29849]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:11:17.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:17 smithi102 ceph-mon[41371]: osdmap e454: 8 total, 8 up, 8 in 2024-09-11T08:11:17.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:17.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:17 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:17.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-85"}]: dispatch 2024-09-11T08:11:17.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:17 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-85"}]: dispatch 2024-09-11T08:11:17.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:17 smithi102 ceph-mon[41371]: pgmap v697: 228 pgs: 3 peering, 225 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 6.0 KiB/s wr, 9 op/s 2024-09-11T08:11:17.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:17 smithi102 ceph-mon[41371]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:11:17.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:17 smithi146 ceph-mon[33158]: osdmap e454: 8 total, 8 up, 8 in 2024-09-11T08:11:17.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:17.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:17 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:17.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-85"}]: dispatch 2024-09-11T08:11:17.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:17 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-85"}]: dispatch 2024-09-11T08:11:17.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:17 smithi146 ceph-mon[33158]: pgmap v697: 228 pgs: 3 peering, 225 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 6.0 KiB/s wr, 9 op/s 2024-09-11T08:11:17.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:17 smithi146 ceph-mon[33158]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:11:18.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:18 smithi102 ceph-mon[29849]: osdmap e455: 8 total, 8 up, 8 in 2024-09-11T08:11:18.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:18 smithi102 ceph-mon[41371]: osdmap e455: 8 total, 8 up, 8 in 2024-09-11T08:11:18.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:18 smithi146 ceph-mon[33158]: osdmap e455: 8 total, 8 up, 8 in 2024-09-11T08:11:19.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:19 smithi102 ceph-mon[29849]: osdmap e456: 8 total, 8 up, 8 in 2024-09-11T08:11:19.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:19.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:19 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:19.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:19 smithi102 ceph-mon[29849]: pgmap v700: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-11T08:11:19.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:19 smithi102 ceph-mon[41371]: osdmap e456: 8 total, 8 up, 8 in 2024-09-11T08:11:19.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:19.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:19 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:19.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:19 smithi102 ceph-mon[41371]: pgmap v700: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-11T08:11:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:19 smithi146 ceph-mon[33158]: osdmap e456: 8 total, 8 up, 8 in 2024-09-11T08:11:19.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:19.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:19 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:19.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:19 smithi146 ceph-mon[33158]: pgmap v700: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-11T08:11:20.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:11:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:11:19] "GET /metrics HTTP/1.1" 200 228685 "" "Prometheus/2.43.0" 2024-09-11T08:11:20.150 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:11:19 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:11:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:20 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:11:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:20 smithi146 ceph-mon[33158]: osdmap e457: 8 total, 8 up, 8 in 2024-09-11T08:11:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:11:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:20 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:20.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:20 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:11:20.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:20 smithi102 ceph-mon[29849]: osdmap e457: 8 total, 8 up, 8 in 2024-09-11T08:11:20.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:11:20.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:20 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:20.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:20 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:11:20.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:20 smithi102 ceph-mon[41371]: osdmap e457: 8 total, 8 up, 8 in 2024-09-11T08:11:20.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:11:20.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:20 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:21.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:21 smithi146 ceph-mon[33158]: osdmap e458: 8 total, 8 up, 8 in 2024-09-11T08:11:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:21 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:21 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:21 smithi146 ceph-mon[33158]: pgmap v703: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:21 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:11:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:21 smithi146 ceph-mon[33158]: osdmap e459: 8 total, 8 up, 8 in 2024-09-11T08:11:21.547 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:21 smithi102 ceph-mon[29849]: osdmap e458: 8 total, 8 up, 8 in 2024-09-11T08:11:21.548 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:21 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:21.548 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:21 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:21.548 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:21 smithi102 ceph-mon[29849]: pgmap v703: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:21.548 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:21 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:11:21.548 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:21 smithi102 ceph-mon[29849]: osdmap e459: 8 total, 8 up, 8 in 2024-09-11T08:11:21.548 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:21 smithi102 ceph-mon[41371]: osdmap e458: 8 total, 8 up, 8 in 2024-09-11T08:11:21.548 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:21 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:21.548 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:21 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:21.548 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:21 smithi102 ceph-mon[41371]: pgmap v703: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:21.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:21 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:11:21.549 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:21 smithi102 ceph-mon[41371]: osdmap e459: 8 total, 8 up, 8 in 2024-09-11T08:11:22.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_tier","val": "test-rados-api-smithi102-80667-89-test-flush"}]: dispatch 2024-09-11T08:11:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:22 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_tier","val": "test-rados-api-smithi102-80667-89-test-flush"}]: dispatch 2024-09-11T08:11:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:22 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:11:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:22 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_tier","val": "test-rados-api-smithi102-80667-89-test-flush"}]': finished 2024-09-11T08:11:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:22 smithi146 ceph-mon[33158]: osdmap e460: 8 total, 8 up, 8 in 2024-09-11T08:11:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:11:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:22 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:11:22.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_tier","val": "test-rados-api-smithi102-80667-89-test-flush"}]: dispatch 2024-09-11T08:11:22.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:22 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_tier","val": "test-rados-api-smithi102-80667-89-test-flush"}]: dispatch 2024-09-11T08:11:22.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:22 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:11:22.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:22 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_tier","val": "test-rados-api-smithi102-80667-89-test-flush"}]': finished 2024-09-11T08:11:22.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:22 smithi102 ceph-mon[41371]: osdmap e460: 8 total, 8 up, 8 in 2024-09-11T08:11:22.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:11:22.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:22 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:11:22.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_tier","val": "test-rados-api-smithi102-80667-89-test-flush"}]: dispatch 2024-09-11T08:11:22.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:22 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_tier","val": "test-rados-api-smithi102-80667-89-test-flush"}]: dispatch 2024-09-11T08:11:22.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:22 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:11:22.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:22 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_tier","val": "test-rados-api-smithi102-80667-89-test-flush"}]': finished 2024-09-11T08:11:22.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:22 smithi102 ceph-mon[29849]: osdmap e460: 8 total, 8 up, 8 in 2024-09-11T08:11:22.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:11:22.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:22 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:11:23.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:23 smithi146 ceph-mon[33158]: pgmap v706: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:23.494 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:23 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:11:23.494 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:23 smithi146 ceph-mon[33158]: osdmap e461: 8 total, 8 up, 8 in 2024-09-11T08:11:23.494 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:11:23.494 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:23 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:11:23.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:23 smithi102 ceph-mon[41371]: pgmap v706: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:23.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:23 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:11:23.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:23 smithi102 ceph-mon[41371]: osdmap e461: 8 total, 8 up, 8 in 2024-09-11T08:11:23.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:11:23.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:23 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:11:23.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:23 smithi102 ceph-mon[29849]: pgmap v706: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:11:23.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:23 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:11:23.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:23 smithi102 ceph-mon[29849]: osdmap e461: 8 total, 8 up, 8 in 2024-09-11T08:11:23.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:11:23.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:23 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:11:24.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:11:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:11:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:11:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:25 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:11:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:25 smithi146 ceph-mon[33158]: osdmap e462: 8 total, 8 up, 8 in 2024-09-11T08:11:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:25 smithi146 ceph-mon[33158]: pgmap v709: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:11:25.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:25 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:11:25.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:25 smithi102 ceph-mon[41371]: osdmap e462: 8 total, 8 up, 8 in 2024-09-11T08:11:25.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:25 smithi102 ceph-mon[41371]: pgmap v709: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:11:25.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:25 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:11:25.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:25 smithi102 ceph-mon[29849]: osdmap e462: 8 total, 8 up, 8 in 2024-09-11T08:11:25.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:25 smithi102 ceph-mon[29849]: pgmap v709: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:11:26.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:26 smithi146 ceph-mon[33158]: osdmap e463: 8 total, 8 up, 8 in 2024-09-11T08:11:26.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:26.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:26 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:26.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-87"}]: dispatch 2024-09-11T08:11:26.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:26 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-87"}]: dispatch 2024-09-11T08:11:26.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:26 smithi102 ceph-mon[41371]: osdmap e463: 8 total, 8 up, 8 in 2024-09-11T08:11:26.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:26.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:26 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:26.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-87"}]: dispatch 2024-09-11T08:11:26.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:26 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-87"}]: dispatch 2024-09-11T08:11:26.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:26 smithi102 ceph-mon[29849]: osdmap e463: 8 total, 8 up, 8 in 2024-09-11T08:11:26.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:26.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:26 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:26.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-87"}]: dispatch 2024-09-11T08:11:26.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:26 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-87"}]: dispatch 2024-09-11T08:11:27.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:27 smithi146 ceph-mon[33158]: osdmap e464: 8 total, 8 up, 8 in 2024-09-11T08:11:27.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:27 smithi146 ceph-mon[33158]: pgmap v712: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:11:27.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:27 smithi102 ceph-mon[41371]: osdmap e464: 8 total, 8 up, 8 in 2024-09-11T08:11:27.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:27 smithi102 ceph-mon[41371]: pgmap v712: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:11:27.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:27 smithi102 ceph-mon[29849]: osdmap e464: 8 total, 8 up, 8 in 2024-09-11T08:11:27.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:27 smithi102 ceph-mon[29849]: pgmap v712: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:11:28.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:28 smithi146 ceph-mon[33158]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:11:28.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:28 smithi146 ceph-mon[33158]: osdmap e465: 8 total, 8 up, 8 in 2024-09-11T08:11:28.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:28.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:28 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:28.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:28 smithi102 ceph-mon[41371]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:11:28.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:28 smithi102 ceph-mon[41371]: osdmap e465: 8 total, 8 up, 8 in 2024-09-11T08:11:28.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:28.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:28 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:28.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:28 smithi102 ceph-mon[29849]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:11:28.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:28 smithi102 ceph-mon[29849]: osdmap e465: 8 total, 8 up, 8 in 2024-09-11T08:11:28.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:28.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:28 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:29.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:29 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:11:29.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:29 smithi102 ceph-mon[29849]: osdmap e466: 8 total, 8 up, 8 in 2024-09-11T08:11:29.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:11:29.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:29.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:29 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:29.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:29 smithi102 ceph-mon[29849]: pgmap v715: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:11:29.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:29 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:11:29.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:29 smithi102 ceph-mon[41371]: osdmap e466: 8 total, 8 up, 8 in 2024-09-11T08:11:29.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:11:29.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:29.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:29 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:29.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:29 smithi102 ceph-mon[41371]: pgmap v715: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:11:29.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:29 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:11:29.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:29 smithi146 ceph-mon[33158]: osdmap e466: 8 total, 8 up, 8 in 2024-09-11T08:11:29.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:11:29.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:29.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:29 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:11:29.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:29 smithi146 ceph-mon[33158]: pgmap v715: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:11:30.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:11:29 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:11:29] "GET /metrics HTTP/1.1" 200 228658 "" "Prometheus/2.43.0" 2024-09-11T08:11:30.184 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:11:29 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:11:30.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:30 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:11:30.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:30 smithi102 ceph-mon[41371]: osdmap e467: 8 total, 8 up, 8 in 2024-09-11T08:11:30.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:30 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:30.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:30 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:30.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:30 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:11:30.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:30 smithi102 ceph-mon[29849]: osdmap e467: 8 total, 8 up, 8 in 2024-09-11T08:11:30.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:30 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:30.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:30 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:30.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:30 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:11:30.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:30 smithi146 ceph-mon[33158]: osdmap e467: 8 total, 8 up, 8 in 2024-09-11T08:11:30.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:30 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:30.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:30 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:31.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:31 smithi102 ceph-mon[29849]: osdmap e468: 8 total, 8 up, 8 in 2024-09-11T08:11:31.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:31 smithi102 ceph-mon[29849]: pgmap v718: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-09-11T08:11:31.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:31 smithi102 ceph-mon[41371]: osdmap e468: 8 total, 8 up, 8 in 2024-09-11T08:11:31.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:31 smithi102 ceph-mon[41371]: pgmap v718: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-09-11T08:11:31.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:31 smithi146 ceph-mon[33158]: osdmap e468: 8 total, 8 up, 8 in 2024-09-11T08:11:31.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:31 smithi146 ceph-mon[33158]: pgmap v718: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-09-11T08:11:32.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:32 smithi102 ceph-mon[41371]: osdmap e469: 8 total, 8 up, 8 in 2024-09-11T08:11:32.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:32.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:32 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:32.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-90"}]: dispatch 2024-09-11T08:11:32.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:32 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-90"}]: dispatch 2024-09-11T08:11:32.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:32 smithi102 ceph-mon[29849]: osdmap e469: 8 total, 8 up, 8 in 2024-09-11T08:11:32.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:32.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:32 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:32.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-90"}]: dispatch 2024-09-11T08:11:32.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:32 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-90"}]: dispatch 2024-09-11T08:11:32.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:32 smithi146 ceph-mon[33158]: osdmap e469: 8 total, 8 up, 8 in 2024-09-11T08:11:32.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:32.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:32 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:32.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-90"}]: dispatch 2024-09-11T08:11:32.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:32 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-90"}]: dispatch 2024-09-11T08:11:33.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:33 smithi102 ceph-mon[41371]: osdmap e470: 8 total, 8 up, 8 in 2024-09-11T08:11:33.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:33 smithi102 ceph-mon[41371]: pgmap v721: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:11:33.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:33 smithi102 ceph-mon[29849]: osdmap e470: 8 total, 8 up, 8 in 2024-09-11T08:11:33.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:33 smithi102 ceph-mon[29849]: pgmap v721: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:11:33.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:33 smithi146 ceph-mon[33158]: osdmap e470: 8 total, 8 up, 8 in 2024-09-11T08:11:33.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:33 smithi146 ceph-mon[33158]: pgmap v721: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:11:34.589 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:34 smithi146 ceph-mon[33158]: osdmap e471: 8 total, 8 up, 8 in 2024-09-11T08:11:34.589 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:34 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:34.589 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:34 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:34.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:34 smithi102 ceph-mon[41371]: osdmap e471: 8 total, 8 up, 8 in 2024-09-11T08:11:34.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:34 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:34.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:34 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:34.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:34 smithi102 ceph-mon[29849]: osdmap e471: 8 total, 8 up, 8 in 2024-09-11T08:11:34.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:34 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:34.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:34 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:34.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:11:34 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:11:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:11:35.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:35 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:11:35.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:35 smithi102 ceph-mon[41371]: osdmap e472: 8 total, 8 up, 8 in 2024-09-11T08:11:35.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:11:35.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:35 smithi102 ceph-mon[41371]: pgmap v724: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:11:35.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:35 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:11:35.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:35 smithi102 ceph-mon[29849]: osdmap e472: 8 total, 8 up, 8 in 2024-09-11T08:11:35.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:11:35.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:35 smithi102 ceph-mon[29849]: pgmap v724: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:11:35.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:35 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:11:35.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:35 smithi146 ceph-mon[33158]: osdmap e472: 8 total, 8 up, 8 in 2024-09-11T08:11:35.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:11:35.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:35 smithi146 ceph-mon[33158]: pgmap v724: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:11:36.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:36 smithi102 ceph-mon[41371]: osdmap e473: 8 total, 8 up, 8 in 2024-09-11T08:11:36.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:36 smithi102 ceph-mon[29849]: osdmap e473: 8 total, 8 up, 8 in 2024-09-11T08:11:36.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:36 smithi146 ceph-mon[33158]: osdmap e473: 8 total, 8 up, 8 in 2024-09-11T08:11:37.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:37 smithi102 ceph-mon[41371]: osdmap e474: 8 total, 8 up, 8 in 2024-09-11T08:11:37.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:37.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:37 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:37.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-92"}]: dispatch 2024-09-11T08:11:37.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:37 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-92"}]: dispatch 2024-09-11T08:11:37.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:37 smithi102 ceph-mon[41371]: pgmap v727: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-11T08:11:37.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:37 smithi102 ceph-mon[29849]: osdmap e474: 8 total, 8 up, 8 in 2024-09-11T08:11:37.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:37.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:37 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:37.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-92"}]: dispatch 2024-09-11T08:11:37.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:37 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-92"}]: dispatch 2024-09-11T08:11:37.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:37 smithi102 ceph-mon[29849]: pgmap v727: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-11T08:11:37.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:37 smithi146 ceph-mon[33158]: osdmap e474: 8 total, 8 up, 8 in 2024-09-11T08:11:37.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:37.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:37 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:11:37.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-92"}]: dispatch 2024-09-11T08:11:37.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:37 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-92"}]: dispatch 2024-09-11T08:11:37.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:37 smithi146 ceph-mon[33158]: pgmap v727: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-11T08:11:38.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:38 smithi102 ceph-mon[41371]: osdmap e475: 8 total, 8 up, 8 in 2024-09-11T08:11:38.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:38 smithi102 ceph-mon[29849]: osdmap e475: 8 total, 8 up, 8 in 2024-09-11T08:11:38.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:38 smithi146 ceph-mon[33158]: osdmap e475: 8 total, 8 up, 8 in 2024-09-11T08:11:39.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:39 smithi102 ceph-mon[29849]: osdmap e476: 8 total, 8 up, 8 in 2024-09-11T08:11:39.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:39 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:39.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:39 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:39.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:39 smithi102 ceph-mon[29849]: pgmap v730: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:11:39.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:39 smithi102 ceph-mon[41371]: osdmap e476: 8 total, 8 up, 8 in 2024-09-11T08:11:39.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:39 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:39.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:39 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:39.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:39 smithi102 ceph-mon[41371]: pgmap v730: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:11:39.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:39 smithi146 ceph-mon[33158]: osdmap e476: 8 total, 8 up, 8 in 2024-09-11T08:11:39.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:39 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:39.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:39 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:11:39.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:39 smithi146 ceph-mon[33158]: pgmap v730: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:11:40.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:11:39 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:11:39] "GET /metrics HTTP/1.1" 200 228658 "" "Prometheus/2.43.0" 2024-09-11T08:11:40.183 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:11:39 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:11:40.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:40 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:11:40.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:40 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:11:40.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:40 smithi102 ceph-mon[29849]: osdmap e477: 8 total, 8 up, 8 in 2024-09-11T08:11:40.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:11:40.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:40 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:40.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:40 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:11:40.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:40 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:11:40.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:40 smithi102 ceph-mon[41371]: osdmap e477: 8 total, 8 up, 8 in 2024-09-11T08:11:40.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:11:40.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:40 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:40.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:40 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:11:40.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:40 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:11:40.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:40 smithi146 ceph-mon[33158]: osdmap e477: 8 total, 8 up, 8 in 2024-09-11T08:11:40.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:11:40.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:40 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:41.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:41 smithi102 ceph-mon[29849]: osdmap e478: 8 total, 8 up, 8 in 2024-09-11T08:11:41.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:41 smithi102 ceph-mon[29849]: pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:11:41.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:41 smithi102 ceph-mon[41371]: osdmap e478: 8 total, 8 up, 8 in 2024-09-11T08:11:41.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:41 smithi102 ceph-mon[41371]: pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:11:41.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:41 smithi146 ceph-mon[33158]: osdmap e478: 8 total, 8 up, 8 in 2024-09-11T08:11:41.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:41 smithi146 ceph-mon[33158]: pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:11:42.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:42 smithi102 ceph-mon[29849]: osdmap e479: 8 total, 8 up, 8 in 2024-09-11T08:11:42.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:42 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:42.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:42 smithi102 ceph-mon[41371]: osdmap e479: 8 total, 8 up, 8 in 2024-09-11T08:11:42.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:42 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:42.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:42 smithi146 ceph-mon[33158]: osdmap e479: 8 total, 8 up, 8 in 2024-09-11T08:11:42.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:42 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:11:43.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.9 KiB/s wr, 3 op/s 2024-09-11T08:11:43.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: Adjusting osd_memory_target on smithi146 to 3248M 2024-09-11T08:11:43.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: Adjusting osd_memory_target on smithi102 to 3248M 2024-09-11T08:11:43.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:11:43.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:11:43.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:43.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:43.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:11:43.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.9 KiB/s wr, 3 op/s 2024-09-11T08:11:43.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.648 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: Adjusting osd_memory_target on smithi146 to 3248M 2024-09-11T08:11:43.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: Adjusting osd_memory_target on smithi102 to 3248M 2024-09-11T08:11:43.649 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:11:43.650 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:11:43.650 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:43.650 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:43.650 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:43 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:11:43.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.9 KiB/s wr, 3 op/s 2024-09-11T08:11:43.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T08:11:43.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: Adjusting osd_memory_target on smithi146 to 3248M 2024-09-11T08:11:43.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: Adjusting osd_memory_target on smithi102 to 3248M 2024-09-11T08:11:43.686 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:11:43.686 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:11:43.686 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:11:43.686 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:11:43.686 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:43 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:11:44.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:44 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:44.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:44 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:44.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:44 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:11:44.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:11:44 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:11:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:11:45.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:45 smithi102 ceph-mon[41371]: pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-11T08:11:45.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:45 smithi102 ceph-mon[29849]: pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-11T08:11:45.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:45 smithi146 ceph-mon[33158]: pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-11T08:11:47.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:47 smithi102 ceph-mon[41371]: pgmap v737: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-11T08:11:47.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:47 smithi102 ceph-mon[41371]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:11:47.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:47 smithi102 ceph-mon[29849]: pgmap v737: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-11T08:11:47.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:47 smithi102 ceph-mon[29849]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:11:47.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:47 smithi146 ceph-mon[33158]: pgmap v737: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-11T08:11:47.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:47 smithi146 ceph-mon[33158]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:11:49.854 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:49 smithi102 ceph-mon[41371]: pgmap v738: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-11T08:11:49.855 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:49 smithi102 ceph-mon[29849]: pgmap v738: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-11T08:11:49.921 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:49 smithi146 ceph-mon[33158]: pgmap v738: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-11T08:11:50.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:11:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:11:49] "GET /metrics HTTP/1.1" 200 228698 "" "Prometheus/2.43.0" 2024-09-11T08:11:50.183 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:11:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:11:50.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:50 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:50.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:50 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:50.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:50 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:11:51.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:51 smithi102 ceph-mon[29849]: pgmap v739: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-09-11T08:11:51.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:51 smithi102 ceph-mon[41371]: pgmap v739: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-09-11T08:11:51.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:51 smithi146 ceph-mon[33158]: pgmap v739: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-09-11T08:11:52.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:52 smithi102 ceph-mon[41371]: osdmap e480: 8 total, 8 up, 8 in 2024-09-11T08:11:52.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:52 smithi102 ceph-mon[29849]: osdmap e480: 8 total, 8 up, 8 in 2024-09-11T08:11:52.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:52 smithi146 ceph-mon[33158]: osdmap e480: 8 total, 8 up, 8 in 2024-09-11T08:11:53.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:53 smithi102 ceph-mon[41371]: pgmap v741: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-09-11T08:11:53.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:53 smithi102 ceph-mon[29849]: pgmap v741: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-09-11T08:11:53.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:53 smithi146 ceph-mon[33158]: pgmap v741: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-09-11T08:11:54.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:11:54 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:11:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:11:55.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:55 smithi102 ceph-mon[29849]: pgmap v742: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-09-11T08:11:55.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:55 smithi102 ceph-mon[41371]: pgmap v742: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-09-11T08:11:55.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:55 smithi146 ceph-mon[33158]: pgmap v742: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-09-11T08:11:58.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:57 smithi102 ceph-mon[41371]: pgmap v743: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:11:58.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:57 smithi102 ceph-mon[41371]: osdmap e481: 8 total, 8 up, 8 in 2024-09-11T08:11:58.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:57 smithi102 ceph-mon[29849]: pgmap v743: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:11:58.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:57 smithi102 ceph-mon[29849]: osdmap e481: 8 total, 8 up, 8 in 2024-09-11T08:11:58.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:57 smithi146 ceph-mon[33158]: pgmap v743: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:11:58.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:57 smithi146 ceph-mon[33158]: osdmap e481: 8 total, 8 up, 8 in 2024-09-11T08:12:00.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:59 smithi102 ceph-mon[41371]: pgmap v745: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:12:00.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:11:59 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:59 smithi102 ceph-mon[29849]: pgmap v745: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:12:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:11:59 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:00.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:11:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:11:59] "GET /metrics HTTP/1.1" 200 228642 "" "Prometheus/2.43.0" 2024-09-11T08:12:00.176 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:59 smithi146 ceph-mon[33158]: pgmap v745: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:12:00.176 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:11:59 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:00.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:11:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:12:00.434 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:12:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=cleanup t=2024-09-11T08:12:00.175833411Z level=info msg="Completed cleanup jobs" duration=12.780692ms 2024-09-11T08:12:01.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:00 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:01.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:00 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:01.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:00 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:02.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:02 smithi102 ceph-mon[41371]: pgmap v746: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:12:02.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:02 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:02.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:02 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:02.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:02 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-94"}]: dispatch 2024-09-11T08:12:02.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:02 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-94"}]: dispatch 2024-09-11T08:12:02.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:02 smithi102 ceph-mon[29849]: pgmap v746: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:12:02.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:02 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:02.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:02 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:02.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:02 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-94"}]: dispatch 2024-09-11T08:12:02.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:02 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-94"}]: dispatch 2024-09-11T08:12:02.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:02 smithi146 ceph-mon[33158]: pgmap v746: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:12:02.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:02 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:02.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:02 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:02.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:02 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-94"}]: dispatch 2024-09-11T08:12:02.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:02 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-94"}]: dispatch 2024-09-11T08:12:03.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:03 smithi102 ceph-mon[41371]: osdmap e482: 8 total, 8 up, 8 in 2024-09-11T08:12:03.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:03 smithi102 ceph-mon[41371]: pgmap v748: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-11T08:12:03.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:03 smithi102 ceph-mon[41371]: osdmap e483: 8 total, 8 up, 8 in 2024-09-11T08:12:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:03 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:03.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:03 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:03.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:03 smithi102 ceph-mon[29849]: osdmap e482: 8 total, 8 up, 8 in 2024-09-11T08:12:03.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:03 smithi102 ceph-mon[29849]: pgmap v748: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-11T08:12:03.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:03 smithi102 ceph-mon[29849]: osdmap e483: 8 total, 8 up, 8 in 2024-09-11T08:12:03.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:03 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:03.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:03 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:03 smithi146 ceph-mon[33158]: osdmap e482: 8 total, 8 up, 8 in 2024-09-11T08:12:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:03 smithi146 ceph-mon[33158]: pgmap v748: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-11T08:12:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:03 smithi146 ceph-mon[33158]: osdmap e483: 8 total, 8 up, 8 in 2024-09-11T08:12:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:03 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:03 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:04.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:12:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:12:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:12:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:04 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:12:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:04 smithi102 ceph-mon[41371]: osdmap e484: 8 total, 8 up, 8 in 2024-09-11T08:12:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:04 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:12:05.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:04 smithi102 ceph-mon[41371]: pgmap v751: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:05.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:04 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:12:05.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:04 smithi102 ceph-mon[29849]: osdmap e484: 8 total, 8 up, 8 in 2024-09-11T08:12:05.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:04 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:12:05.398 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:04 smithi102 ceph-mon[29849]: pgmap v751: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:04 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:12:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:04 smithi146 ceph-mon[33158]: osdmap e484: 8 total, 8 up, 8 in 2024-09-11T08:12:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:04 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:12:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:04 smithi146 ceph-mon[33158]: pgmap v751: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:06.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:05 smithi102 ceph-mon[41371]: osdmap e485: 8 total, 8 up, 8 in 2024-09-11T08:12:06.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:05 smithi102 ceph-mon[29849]: osdmap e485: 8 total, 8 up, 8 in 2024-09-11T08:12:06.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:05 smithi146 ceph-mon[33158]: osdmap e485: 8 total, 8 up, 8 in 2024-09-11T08:12:07.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:06 smithi102 ceph-mon[41371]: pgmap v753: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-11T08:12:07.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:06 smithi102 ceph-mon[29849]: pgmap v753: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-11T08:12:07.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:06 smithi146 ceph-mon[33158]: pgmap v753: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-11T08:12:09.850 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:09 smithi102 ceph-mon[29849]: pgmap v754: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:12:09.850 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:09 smithi102 ceph-mon[41371]: pgmap v754: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:12:09.923 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:09 smithi146 ceph-mon[33158]: pgmap v754: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 1023 B/s wr, 2 op/s 2024-09-11T08:12:10.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:12:09 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:12:09] "GET /metrics HTTP/1.1" 200 228642 "" "Prometheus/2.43.0" 2024-09-11T08:12:10.183 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:12:09 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:12:10.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:10 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:10.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:10 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:10.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:10 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:11.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:11 smithi102 ceph-mon[29849]: pgmap v755: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 813 B/s wr, 2 op/s 2024-09-11T08:12:11.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:11 smithi102 ceph-mon[41371]: pgmap v755: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 813 B/s wr, 2 op/s 2024-09-11T08:12:11.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:11 smithi146 ceph-mon[33158]: pgmap v755: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 813 B/s wr, 2 op/s 2024-09-11T08:12:13.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:13 smithi102 ceph-mon[41371]: pgmap v756: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 718 B/s wr, 2 op/s 2024-09-11T08:12:13.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:13 smithi102 ceph-mon[29849]: pgmap v756: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 718 B/s wr, 2 op/s 2024-09-11T08:12:13.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:13 smithi146 ceph-mon[33158]: pgmap v756: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 718 B/s wr, 2 op/s 2024-09-11T08:12:14.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:14 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:14.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:14 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:14.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:14 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:14.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:12:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:12:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:12:15.395 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 08:12:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-2[58066]: 2024-09-11T08:12:15.008+0000 7f33daddc640 -1 snap_mapper.add_oid found existing snaps mapped on 64:67868683:test-rados-api-smithi102-80667-97::foo:31, removing 2024-09-11T08:12:15.433 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:12:15 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:12:15.008+0000 7fc685396640 -1 snap_mapper.add_oid found existing snaps mapped on 64:67868683:test-rados-api-smithi102-80667-97::foo:31, removing 2024-09-11T08:12:15.434 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:12:15 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:12:15.007+0000 7f62fd1bd640 -1 snap_mapper.add_oid found existing snaps mapped on 64:67868683:test-rados-api-smithi102-80667-97::foo:31, removing 2024-09-11T08:12:15.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:15 smithi102 ceph-mon[29849]: pgmap v757: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 614 B/s wr, 2 op/s 2024-09-11T08:12:15.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:15 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:15.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:15 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:15.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:15 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-96"}]: dispatch 2024-09-11T08:12:15.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:15 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-96"}]: dispatch 2024-09-11T08:12:15.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:15 smithi102 ceph-mon[41371]: pgmap v757: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 614 B/s wr, 2 op/s 2024-09-11T08:12:15.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:15 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:15.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:15 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:15.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:15 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-96"}]: dispatch 2024-09-11T08:12:15.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:15 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-96"}]: dispatch 2024-09-11T08:12:15.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:15 smithi146 ceph-mon[33158]: pgmap v757: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 614 B/s wr, 2 op/s 2024-09-11T08:12:15.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:15 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:15.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:15 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:15.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:15 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-96"}]: dispatch 2024-09-11T08:12:15.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:15 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-96"}]: dispatch 2024-09-11T08:12:16.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:16 smithi102 ceph-mon[29849]: osdmap e486: 8 total, 8 up, 8 in 2024-09-11T08:12:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:16 smithi102 ceph-mon[41371]: osdmap e486: 8 total, 8 up, 8 in 2024-09-11T08:12:16.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:16 smithi146 ceph-mon[33158]: osdmap e486: 8 total, 8 up, 8 in 2024-09-11T08:12:17.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:17 smithi102 ceph-mon[41371]: pgmap v759: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-09-11T08:12:17.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:17 smithi102 ceph-mon[41371]: osdmap e487: 8 total, 8 up, 8 in 2024-09-11T08:12:17.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:17.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:17 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:17.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:17 smithi102 ceph-mon[29849]: pgmap v759: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-09-11T08:12:17.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:17 smithi102 ceph-mon[29849]: osdmap e487: 8 total, 8 up, 8 in 2024-09-11T08:12:17.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:17.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:17 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:17.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:17 smithi146 ceph-mon[33158]: pgmap v759: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 op/s 2024-09-11T08:12:17.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:17 smithi146 ceph-mon[33158]: osdmap e487: 8 total, 8 up, 8 in 2024-09-11T08:12:17.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:17.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:17 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:18.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:18 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:12:18.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:18 smithi102 ceph-mon[41371]: osdmap e488: 8 total, 8 up, 8 in 2024-09-11T08:12:18.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:12:18.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:18 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:12:18.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:18 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:12:18.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:18 smithi102 ceph-mon[29849]: osdmap e488: 8 total, 8 up, 8 in 2024-09-11T08:12:18.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:12:18.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:18 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:12:18.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:18 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:12:18.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:18 smithi146 ceph-mon[33158]: osdmap e488: 8 total, 8 up, 8 in 2024-09-11T08:12:18.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:12:18.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:18 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:12:19.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:19 smithi146 ceph-mon[33158]: pgmap v762: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:12:19.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:19 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:12:19.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:19 smithi146 ceph-mon[33158]: osdmap e489: 8 total, 8 up, 8 in 2024-09-11T08:12:19.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-98", "mode": "writeback"}]: dispatch 2024-09-11T08:12:19.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:19 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-98", "mode": "writeback"}]: dispatch 2024-09-11T08:12:20.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:12:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:12:19] "GET /metrics HTTP/1.1" 200 228634 "" "Prometheus/2.43.0" 2024-09-11T08:12:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:19 smithi102 ceph-mon[29849]: pgmap v762: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:12:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:19 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:12:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:19 smithi102 ceph-mon[29849]: osdmap e489: 8 total, 8 up, 8 in 2024-09-11T08:12:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-98", "mode": "writeback"}]: dispatch 2024-09-11T08:12:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:19 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-98", "mode": "writeback"}]: dispatch 2024-09-11T08:12:20.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:19 smithi102 ceph-mon[41371]: pgmap v762: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:12:20.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:19 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:12:20.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:19 smithi102 ceph-mon[41371]: osdmap e489: 8 total, 8 up, 8 in 2024-09-11T08:12:20.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-98", "mode": "writeback"}]: dispatch 2024-09-11T08:12:20.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:19 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-98", "mode": "writeback"}]: dispatch 2024-09-11T08:12:20.434 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:12:19 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:12:20.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:20 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:12:20.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:20 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-98", "mode": "writeback"}]': finished 2024-09-11T08:12:20.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:20 smithi146 ceph-mon[33158]: osdmap e490: 8 total, 8 up, 8 in 2024-09-11T08:12:20.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-98"}]: dispatch 2024-09-11T08:12:20.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:20 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-98"}]: dispatch 2024-09-11T08:12:20.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:20 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:20.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:20 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-98"}]': finished 2024-09-11T08:12:20.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:20 smithi146 ceph-mon[33158]: osdmap e491: 8 total, 8 up, 8 in 2024-09-11T08:12:20.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:12:20.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:20 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:12:21.046 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:20 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:12:21.046 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:20 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-98", "mode": "writeback"}]': finished 2024-09-11T08:12:21.046 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:20 smithi102 ceph-mon[29849]: osdmap e490: 8 total, 8 up, 8 in 2024-09-11T08:12:21.046 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-98"}]: dispatch 2024-09-11T08:12:21.046 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:20 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-98"}]: dispatch 2024-09-11T08:12:21.046 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:20 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:21.047 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:20 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-98"}]': finished 2024-09-11T08:12:21.047 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:20 smithi102 ceph-mon[29849]: osdmap e491: 8 total, 8 up, 8 in 2024-09-11T08:12:21.047 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:12:21.047 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:20 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:12:21.047 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:20 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:12:21.047 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:20 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-98", "mode": "writeback"}]': finished 2024-09-11T08:12:21.047 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:20 smithi102 ceph-mon[41371]: osdmap e490: 8 total, 8 up, 8 in 2024-09-11T08:12:21.047 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-98"}]: dispatch 2024-09-11T08:12:21.047 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:20 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-98"}]: dispatch 2024-09-11T08:12:21.047 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:20 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:21.048 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:20 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-98"}]': finished 2024-09-11T08:12:21.048 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:20 smithi102 ceph-mon[41371]: osdmap e491: 8 total, 8 up, 8 in 2024-09-11T08:12:21.048 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:12:21.048 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:20 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:12:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:21 smithi146 ceph-mon[33158]: pgmap v765: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:12:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:21 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:12:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:21 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:12:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:21 smithi146 ceph-mon[33158]: osdmap e492: 8 total, 8 up, 8 in 2024-09-11T08:12:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:21 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-11T08:12:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:21 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-11T08:12:22.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:21 smithi102 ceph-mon[41371]: pgmap v765: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:12:22.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:21 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:12:22.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:21 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:12:22.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:21 smithi102 ceph-mon[41371]: osdmap e492: 8 total, 8 up, 8 in 2024-09-11T08:12:22.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:21 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-11T08:12:22.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:21 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-11T08:12:22.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:21 smithi102 ceph-mon[29849]: pgmap v765: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:12:22.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:21 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:12:22.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:21 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:12:22.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:21 smithi102 ceph-mon[29849]: osdmap e492: 8 total, 8 up, 8 in 2024-09-11T08:12:22.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:21 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-11T08:12:22.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:21 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-11T08:12:23.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:23 smithi146 ceph-mon[33158]: pgmap v768: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:12:23.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:23 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_count","val": "1"}]': finished 2024-09-11T08:12:23.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:23 smithi146 ceph-mon[33158]: osdmap e493: 8 total, 8 up, 8 in 2024-09-11T08:12:23.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:12:23.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:23 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:12:24.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:23 smithi102 ceph-mon[41371]: pgmap v768: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:12:24.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:23 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_count","val": "1"}]': finished 2024-09-11T08:12:24.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:23 smithi102 ceph-mon[41371]: osdmap e493: 8 total, 8 up, 8 in 2024-09-11T08:12:24.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:12:24.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:23 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:12:24.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:23 smithi102 ceph-mon[29849]: pgmap v768: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:12:24.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:23 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_count","val": "1"}]': finished 2024-09-11T08:12:24.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:23 smithi102 ceph-mon[29849]: osdmap e493: 8 total, 8 up, 8 in 2024-09-11T08:12:24.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:12:24.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:23 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:12:24.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:12:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:12:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:12:24.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:24 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:12:24.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:24 smithi146 ceph-mon[33158]: osdmap e494: 8 total, 8 up, 8 in 2024-09-11T08:12:24.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-11T08:12:24.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:24 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-11T08:12:25.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:24 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:12:25.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:24 smithi102 ceph-mon[41371]: osdmap e494: 8 total, 8 up, 8 in 2024-09-11T08:12:25.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-11T08:12:25.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:24 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-11T08:12:25.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:24 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:12:25.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:24 smithi102 ceph-mon[29849]: osdmap e494: 8 total, 8 up, 8 in 2024-09-11T08:12:25.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-11T08:12:25.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:24 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-11T08:12:25.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:25 smithi146 ceph-mon[33158]: pgmap v771: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:25.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:25 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "target_max_objects","val": "250"}]': finished 2024-09-11T08:12:25.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:25 smithi146 ceph-mon[33158]: osdmap e495: 8 total, 8 up, 8 in 2024-09-11T08:12:25.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:25.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:25 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:26.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:25 smithi102 ceph-mon[29849]: pgmap v771: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:26.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:25 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "target_max_objects","val": "250"}]': finished 2024-09-11T08:12:26.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:25 smithi102 ceph-mon[29849]: osdmap e495: 8 total, 8 up, 8 in 2024-09-11T08:12:26.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:26.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:25 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:26.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:25 smithi102 ceph-mon[41371]: pgmap v771: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:26.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:25 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-98","var": "target_max_objects","val": "250"}]': finished 2024-09-11T08:12:26.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:25 smithi102 ceph-mon[41371]: osdmap e495: 8 total, 8 up, 8 in 2024-09-11T08:12:26.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:26.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:25 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:26.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:26 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:12:26.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:26 smithi146 ceph-mon[33158]: osdmap e496: 8 total, 8 up, 8 in 2024-09-11T08:12:26.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-98"}]: dispatch 2024-09-11T08:12:26.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:26 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-98"}]: dispatch 2024-09-11T08:12:27.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:26 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:12:27.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:26 smithi102 ceph-mon[29849]: osdmap e496: 8 total, 8 up, 8 in 2024-09-11T08:12:27.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-98"}]: dispatch 2024-09-11T08:12:27.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:26 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-98"}]: dispatch 2024-09-11T08:12:27.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:26 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:12:27.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:26 smithi102 ceph-mon[41371]: osdmap e496: 8 total, 8 up, 8 in 2024-09-11T08:12:27.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-98"}]: dispatch 2024-09-11T08:12:27.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:26 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-98"}]: dispatch 2024-09-11T08:12:27.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:27 smithi146 ceph-mon[33158]: pgmap v774: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:12:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:27 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-98"}]': finished 2024-09-11T08:12:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:27 smithi146 ceph-mon[33158]: osdmap e497: 8 total, 8 up, 8 in 2024-09-11T08:12:28.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:27 smithi102 ceph-mon[41371]: pgmap v774: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:12:28.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:27 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-98"}]': finished 2024-09-11T08:12:28.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:27 smithi102 ceph-mon[41371]: osdmap e497: 8 total, 8 up, 8 in 2024-09-11T08:12:28.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:27 smithi102 ceph-mon[29849]: pgmap v774: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:12:28.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:27 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-98"}]': finished 2024-09-11T08:12:28.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:27 smithi102 ceph-mon[29849]: osdmap e497: 8 total, 8 up, 8 in 2024-09-11T08:12:29.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:28 smithi102 ceph-mon[41371]: osdmap e498: 8 total, 8 up, 8 in 2024-09-11T08:12:29.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:28 smithi102 ceph-mon[29849]: osdmap e498: 8 total, 8 up, 8 in 2024-09-11T08:12:29.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:28 smithi146 ceph-mon[33158]: osdmap e498: 8 total, 8 up, 8 in 2024-09-11T08:12:30.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:12:29 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:12:29] "GET /metrics HTTP/1.1" 200 228618 "" "Prometheus/2.43.0" 2024-09-11T08:12:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:29 smithi102 ceph-mon[41371]: pgmap v777: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:29 smithi102 ceph-mon[41371]: osdmap e499: 8 total, 8 up, 8 in 2024-09-11T08:12:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:29 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:29 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:30.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:29 smithi102 ceph-mon[29849]: pgmap v777: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:30.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:29 smithi102 ceph-mon[29849]: osdmap e499: 8 total, 8 up, 8 in 2024-09-11T08:12:30.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:30.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:29 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:30.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:29 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:30.183 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:12:29 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:12:30.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:29 smithi146 ceph-mon[33158]: pgmap v777: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:30.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:29 smithi146 ceph-mon[33158]: osdmap e499: 8 total, 8 up, 8 in 2024-09-11T08:12:30.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:30.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:29 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:30.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:29 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:31.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:30 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:12:31.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:30 smithi102 ceph-mon[41371]: osdmap e500: 8 total, 8 up, 8 in 2024-09-11T08:12:31.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:30 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:12:31.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:30 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:12:31.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:30 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:31.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:30 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:12:31.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:30 smithi102 ceph-mon[41371]: osdmap e501: 8 total, 8 up, 8 in 2024-09-11T08:12:31.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:30 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-100"}]: dispatch 2024-09-11T08:12:31.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:30 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-100"}]: dispatch 2024-09-11T08:12:31.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:30 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:12:31.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:30 smithi102 ceph-mon[29849]: osdmap e500: 8 total, 8 up, 8 in 2024-09-11T08:12:31.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:30 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:12:31.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:30 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:12:31.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:30 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:31.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:30 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:12:31.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:30 smithi102 ceph-mon[29849]: osdmap e501: 8 total, 8 up, 8 in 2024-09-11T08:12:31.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:30 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-100"}]: dispatch 2024-09-11T08:12:31.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:30 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-100"}]: dispatch 2024-09-11T08:12:31.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:30 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:12:31.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:30 smithi146 ceph-mon[33158]: osdmap e500: 8 total, 8 up, 8 in 2024-09-11T08:12:31.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:30 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:12:31.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:30 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:12:31.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:30 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:31.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:30 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:12:31.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:30 smithi146 ceph-mon[33158]: osdmap e501: 8 total, 8 up, 8 in 2024-09-11T08:12:31.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:30 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-100"}]: dispatch 2024-09-11T08:12:31.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:30 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-100"}]: dispatch 2024-09-11T08:12:32.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:31 smithi102 ceph-mon[29849]: pgmap v780: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:32.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:31 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-100"}]': finished 2024-09-11T08:12:32.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:31 smithi102 ceph-mon[29849]: osdmap e502: 8 total, 8 up, 8 in 2024-09-11T08:12:32.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:31 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-100", "mode": "writeback"}]: dispatch 2024-09-11T08:12:32.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:31 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-100", "mode": "writeback"}]: dispatch 2024-09-11T08:12:32.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:31 smithi102 ceph-mon[41371]: pgmap v780: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:32.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:31 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-100"}]': finished 2024-09-11T08:12:32.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:31 smithi102 ceph-mon[41371]: osdmap e502: 8 total, 8 up, 8 in 2024-09-11T08:12:32.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:31 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-100", "mode": "writeback"}]: dispatch 2024-09-11T08:12:32.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:31 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-100", "mode": "writeback"}]: dispatch 2024-09-11T08:12:32.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:31 smithi146 ceph-mon[33158]: pgmap v780: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:32.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:31 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-6", "overlaypool": "test-rados-api-smithi102-80667-100"}]': finished 2024-09-11T08:12:32.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:31 smithi146 ceph-mon[33158]: osdmap e502: 8 total, 8 up, 8 in 2024-09-11T08:12:32.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:31 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-100", "mode": "writeback"}]: dispatch 2024-09-11T08:12:32.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:31 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-100", "mode": "writeback"}]: dispatch 2024-09-11T08:12:33.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:32 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:12:33.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:32 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-100", "mode": "writeback"}]': finished 2024-09-11T08:12:33.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:32 smithi102 ceph-mon[29849]: osdmap e503: 8 total, 8 up, 8 in 2024-09-11T08:12:33.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:12:33.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:32 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:12:33.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:32 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:12:33.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:32 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-100", "mode": "writeback"}]': finished 2024-09-11T08:12:33.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:32 smithi102 ceph-mon[41371]: osdmap e503: 8 total, 8 up, 8 in 2024-09-11T08:12:33.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:12:33.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:32 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:12:33.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:32 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:12:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:32 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-100", "mode": "writeback"}]': finished 2024-09-11T08:12:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:32 smithi146 ceph-mon[33158]: osdmap e503: 8 total, 8 up, 8 in 2024-09-11T08:12:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:12:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:32 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:12:34.085 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:33 smithi146 ceph-mon[33158]: pgmap v783: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:34.085 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:33 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:12:34.085 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:33 smithi146 ceph-mon[33158]: osdmap e504: 8 total, 8 up, 8 in 2024-09-11T08:12:34.085 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:12:34.085 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:33 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:12:34.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:33 smithi102 ceph-mon[29849]: pgmap v783: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:34.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:33 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:12:34.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:33 smithi102 ceph-mon[29849]: osdmap e504: 8 total, 8 up, 8 in 2024-09-11T08:12:34.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:12:34.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:33 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:12:34.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:33 smithi102 ceph-mon[41371]: pgmap v783: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:34.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:33 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:12:34.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:33 smithi102 ceph-mon[41371]: osdmap e504: 8 total, 8 up, 8 in 2024-09-11T08:12:34.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:12:34.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:33 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:12:34.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:12:34 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:12:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:12:36.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:35 smithi102 ceph-mon[29849]: pgmap v786: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:36.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:35 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:12:36.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:35 smithi102 ceph-mon[29849]: osdmap e505: 8 total, 8 up, 8 in 2024-09-11T08:12:36.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:12:36.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:35 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:12:36.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:35 smithi102 ceph-mon[41371]: pgmap v786: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:36.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:35 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:12:36.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:35 smithi102 ceph-mon[41371]: osdmap e505: 8 total, 8 up, 8 in 2024-09-11T08:12:36.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:12:36.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:35 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:12:36.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:35 smithi146 ceph-mon[33158]: pgmap v786: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:36.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:35 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:12:36.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:35 smithi146 ceph-mon[33158]: osdmap e505: 8 total, 8 up, 8 in 2024-09-11T08:12:36.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:12:36.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:35 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:12:37.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:36 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:12:37.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:36 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:12:37.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:36 smithi102 ceph-mon[29849]: osdmap e506: 8 total, 8 up, 8 in 2024-09-11T08:12:37.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:36 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-11T08:12:37.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:36 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-11T08:12:37.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:36 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-11T08:12:37.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:36 smithi102 ceph-mon[29849]: osdmap e507: 8 total, 8 up, 8 in 2024-09-11T08:12:37.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:36 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:12:37.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:36 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:12:37.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:36 smithi102 ceph-mon[41371]: osdmap e506: 8 total, 8 up, 8 in 2024-09-11T08:12:37.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:36 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-11T08:12:37.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:36 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-11T08:12:37.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:36 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-11T08:12:37.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:36 smithi102 ceph-mon[41371]: osdmap e507: 8 total, 8 up, 8 in 2024-09-11T08:12:37.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:36 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:12:37.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:36 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:12:37.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:36 smithi146 ceph-mon[33158]: osdmap e506: 8 total, 8 up, 8 in 2024-09-11T08:12:37.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:36 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-11T08:12:37.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:36 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-11T08:12:37.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:36 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-11T08:12:37.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:36 smithi146 ceph-mon[33158]: osdmap e507: 8 total, 8 up, 8 in 2024-09-11T08:12:38.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:37 smithi102 ceph-mon[41371]: pgmap v789: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:38.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:38.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:37 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:38.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:37 smithi102 ceph-mon[29849]: pgmap v789: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:38.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:38.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:37 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:38.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:37 smithi146 ceph-mon[33158]: pgmap v789: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:38.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:38.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:37 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:39.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:38 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:12:39.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:38 smithi102 ceph-mon[41371]: osdmap e508: 8 total, 8 up, 8 in 2024-09-11T08:12:39.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-100"}]: dispatch 2024-09-11T08:12:39.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:38 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-100"}]: dispatch 2024-09-11T08:12:39.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:38 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:12:39.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:38 smithi102 ceph-mon[29849]: osdmap e508: 8 total, 8 up, 8 in 2024-09-11T08:12:39.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-100"}]: dispatch 2024-09-11T08:12:39.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:38 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-100"}]: dispatch 2024-09-11T08:12:39.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:38 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]': finished 2024-09-11T08:12:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:38 smithi146 ceph-mon[33158]: osdmap e508: 8 total, 8 up, 8 in 2024-09-11T08:12:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-100"}]: dispatch 2024-09-11T08:12:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:38 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-100"}]: dispatch 2024-09-11T08:12:40.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:12:39 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:12:39] "GET /metrics HTTP/1.1" 200 228618 "" "Prometheus/2.43.0" 2024-09-11T08:12:40.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:39 smithi102 ceph-mon[41371]: pgmap v792: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:40.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:39 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-100"}]': finished 2024-09-11T08:12:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:39 smithi102 ceph-mon[41371]: osdmap e509: 8 total, 8 up, 8 in 2024-09-11T08:12:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:39 smithi102 ceph-mon[29849]: pgmap v792: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:39 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-100"}]': finished 2024-09-11T08:12:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:39 smithi102 ceph-mon[29849]: osdmap e509: 8 total, 8 up, 8 in 2024-09-11T08:12:40.434 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:12:39 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:12:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:39 smithi146 ceph-mon[33158]: pgmap v792: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:39 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-100"}]': finished 2024-09-11T08:12:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:39 smithi146 ceph-mon[33158]: osdmap e509: 8 total, 8 up, 8 in 2024-09-11T08:12:41.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:40 smithi102 ceph-mon[41371]: osdmap e510: 8 total, 8 up, 8 in 2024-09-11T08:12:41.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:40 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:41.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:40 smithi102 ceph-mon[41371]: osdmap e511: 8 total, 8 up, 8 in 2024-09-11T08:12:41.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:41.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:40 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:40 smithi102 ceph-mon[29849]: osdmap e510: 8 total, 8 up, 8 in 2024-09-11T08:12:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:40 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:40 smithi102 ceph-mon[29849]: osdmap e511: 8 total, 8 up, 8 in 2024-09-11T08:12:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:40 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:40 smithi146 ceph-mon[33158]: osdmap e510: 8 total, 8 up, 8 in 2024-09-11T08:12:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:40 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:40 smithi146 ceph-mon[33158]: osdmap e511: 8 total, 8 up, 8 in 2024-09-11T08:12:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:40 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:42.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:41 smithi102 ceph-mon[41371]: pgmap v795: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:12:42.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:41 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:12:42.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:41 smithi102 ceph-mon[41371]: osdmap e512: 8 total, 8 up, 8 in 2024-09-11T08:12:42.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:12:42.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:12:42.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:41 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:12:42.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:41 smithi102 ceph-mon[29849]: pgmap v795: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:12:42.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:41 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:12:42.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:41 smithi102 ceph-mon[29849]: osdmap e512: 8 total, 8 up, 8 in 2024-09-11T08:12:42.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:12:42.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:12:42.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:41 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:12:42.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:41 smithi146 ceph-mon[33158]: pgmap v795: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:12:42.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:41 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:12:42.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:41 smithi146 ceph-mon[33158]: osdmap e512: 8 total, 8 up, 8 in 2024-09-11T08:12:42.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-11T08:12:42.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:12:42.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:41 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-11T08:12:43.335 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:42 smithi102 ceph-mon[41371]: pgmap v798: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:12:43.336 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:42 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:12:43.336 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:42 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:12:43.336 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:42 smithi102 ceph-mon[41371]: osdmap e513: 8 total, 8 up, 8 in 2024-09-11T08:12:43.336 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:12:43.336 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:42 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:12:43.336 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:42 smithi102 ceph-mon[29849]: pgmap v798: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:12:43.336 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:42 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:12:43.337 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:42 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:12:43.337 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:42 smithi102 ceph-mon[29849]: osdmap e513: 8 total, 8 up, 8 in 2024-09-11T08:12:43.337 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:12:43.337 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:42 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:12:43.380 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:42 smithi146 ceph-mon[33158]: pgmap v798: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:12:43.380 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:42 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:12:43.380 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:42 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-11T08:12:43.380 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:42 smithi146 ceph-mon[33158]: osdmap e513: 8 total, 8 up, 8 in 2024-09-11T08:12:43.380 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:12:43.381 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:42 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-11T08:12:44.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:12:44 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:12:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:12:44.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:44 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:12:44.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:44 smithi146 ceph-mon[33158]: osdmap e514: 8 total, 8 up, 8 in 2024-09-11T08:12:44.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:44 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:12:44.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:44 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:12:44.935 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:44 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:45.035 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:44 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:12:45.035 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:44 smithi102 ceph-mon[29849]: osdmap e514: 8 total, 8 up, 8 in 2024-09-11T08:12:45.035 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:44 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:12:45.035 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:44 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:12:45.036 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:44 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:45.036 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:44 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-11T08:12:45.036 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:44 smithi102 ceph-mon[41371]: osdmap e514: 8 total, 8 up, 8 in 2024-09-11T08:12:45.036 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:44 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:12:45.036 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:44 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-11T08:12:45.036 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:44 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:12:46.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:45 smithi102 ceph-mon[29849]: pgmap v801: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:46.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:45 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:12:46.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:45 smithi102 ceph-mon[29849]: osdmap e515: 8 total, 8 up, 8 in 2024-09-11T08:12:46.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:45 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:46.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:45 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:46.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:45 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-102"}]: dispatch 2024-09-11T08:12:46.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:45 smithi102 ceph-mon[29849]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-102"}]: dispatch 2024-09-11T08:12:46.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:45 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:12:46.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:45 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:12:46.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:45 smithi102 ceph-mon[41371]: pgmap v801: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:46.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:45 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:12:46.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:45 smithi102 ceph-mon[41371]: osdmap e515: 8 total, 8 up, 8 in 2024-09-11T08:12:46.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:45 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:46.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:45 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:46.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:45 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-102"}]: dispatch 2024-09-11T08:12:46.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:45 smithi102 ceph-mon[41371]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-102"}]: dispatch 2024-09-11T08:12:46.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:45 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:12:46.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:45 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:12:46.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:45 smithi146 ceph-mon[33158]: pgmap v801: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:46.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:45 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-11T08:12:46.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:45 smithi146 ceph-mon[33158]: osdmap e515: 8 total, 8 up, 8 in 2024-09-11T08:12:46.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:45 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:46.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:45 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-6"}]: dispatch 2024-09-11T08:12:46.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:45 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3853348599' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-102"}]: dispatch 2024-09-11T08:12:46.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:45 smithi146 ceph-mon[33158]: from='client.26077 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-6", "tierpool": "test-rados-api-smithi102-80667-102"}]: dispatch 2024-09-11T08:12:46.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:45 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:12:46.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:45 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:12:47.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:46 smithi102 ceph-mon[41371]: osdmap e516: 8 total, 8 up, 8 in 2024-09-11T08:12:47.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:46 smithi102 ceph-mon[29849]: osdmap e516: 8 total, 8 up, 8 in 2024-09-11T08:12:47.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:46 smithi146 ceph-mon[33158]: osdmap e516: 8 total, 8 up, 8 in 2024-09-11T08:12:48.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:47 smithi102 ceph-mon[41371]: pgmap v804: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:12:48.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:47 smithi102 ceph-mon[41371]: osdmap e517: 8 total, 8 up, 8 in 2024-09-11T08:12:48.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi102-80667-104"}]: dispatch 2024-09-11T08:12:48.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi102-80667-104"}]: dispatch 2024-09-11T08:12:48.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi102-80667-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:12:48.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:47 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:12:48.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:47 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:12:48.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:47 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:12:48.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:47 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:12:48.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:47 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:12:48.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:47 smithi102 ceph-mon[29849]: pgmap v804: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:12:48.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:47 smithi102 ceph-mon[29849]: osdmap e517: 8 total, 8 up, 8 in 2024-09-11T08:12:48.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi102-80667-104"}]: dispatch 2024-09-11T08:12:48.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi102-80667-104"}]: dispatch 2024-09-11T08:12:48.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi102-80667-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:12:48.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:47 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:12:48.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:47 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:12:48.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:47 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:12:48.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:47 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:12:48.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:47 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:12:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:47 smithi146 ceph-mon[33158]: pgmap v804: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:12:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:47 smithi146 ceph-mon[33158]: osdmap e517: 8 total, 8 up, 8 in 2024-09-11T08:12:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi102-80667-104"}]: dispatch 2024-09-11T08:12:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi102-80667-104"}]: dispatch 2024-09-11T08:12:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi102-80667-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:12:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:47 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:12:48.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:47 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:12:48.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:47 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:12:48.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:47 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:12:48.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:47 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:12:49.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi102-80667-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:12:49.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:48 smithi102 ceph-mon[29849]: osdmap e518: 8 total, 8 up, 8 in 2024-09-11T08:12:49.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi102-80667-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi102-80667-104"}]: dispatch 2024-09-11T08:12:49.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi102-80667-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:12:49.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:48 smithi102 ceph-mon[41371]: osdmap e518: 8 total, 8 up, 8 in 2024-09-11T08:12:49.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi102-80667-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi102-80667-104"}]: dispatch 2024-09-11T08:12:49.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi102-80667-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:12:49.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:48 smithi146 ceph-mon[33158]: osdmap e518: 8 total, 8 up, 8 in 2024-09-11T08:12:49.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi102-80667-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi102-80667-104"}]: dispatch 2024-09-11T08:12:50.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:12:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:12:49] "GET /metrics HTTP/1.1" 200 228646 "" "Prometheus/2.43.0" 2024-09-11T08:12:50.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:49 smithi102 ceph-mon[41371]: pgmap v807: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:50.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:49 smithi102 ceph-mon[41371]: osdmap e519: 8 total, 8 up, 8 in 2024-09-11T08:12:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:49 smithi102 ceph-mon[29849]: pgmap v807: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:49 smithi102 ceph-mon[29849]: osdmap e519: 8 total, 8 up, 8 in 2024-09-11T08:12:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:49 smithi146 ceph-mon[33158]: pgmap v807: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:50.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:49 smithi146 ceph-mon[33158]: osdmap e519: 8 total, 8 up, 8 in 2024-09-11T08:12:50.184 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:12:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:12:51.046 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:50 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi102-80667-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi102-80667-104"}]': finished 2024-09-11T08:12:51.046 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:50 smithi102 ceph-mon[41371]: osdmap e520: 8 total, 8 up, 8 in 2024-09-11T08:12:51.046 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:50 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:51.046 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:50 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi102-80667-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi102-80667-104"}]': finished 2024-09-11T08:12:51.046 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:50 smithi102 ceph-mon[29849]: osdmap e520: 8 total, 8 up, 8 in 2024-09-11T08:12:51.046 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:50 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:51.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:50 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi102-80667-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi102-80667-104"}]': finished 2024-09-11T08:12:51.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:50 smithi146 ceph-mon[33158]: osdmap e520: 8 total, 8 up, 8 in 2024-09-11T08:12:51.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:50 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:12:52.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:51 smithi102 ceph-mon[41371]: pgmap v810: 172 pgs: 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:52.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:51 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:12:52.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:51 smithi102 ceph-mon[41371]: osdmap e521: 8 total, 8 up, 8 in 2024-09-11T08:12:52.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:51 smithi102 ceph-mon[29849]: pgmap v810: 172 pgs: 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:52.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:51 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:12:52.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:51 smithi102 ceph-mon[29849]: osdmap e521: 8 total, 8 up, 8 in 2024-09-11T08:12:52.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:51 smithi146 ceph-mon[33158]: pgmap v810: 172 pgs: 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:52.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:51 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:12:52.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:51 smithi146 ceph-mon[33158]: osdmap e521: 8 total, 8 up, 8 in 2024-09-11T08:12:53.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:53 smithi102 ceph-mon[29849]: osdmap e522: 8 total, 8 up, 8 in 2024-09-11T08:12:53.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:53 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3279590380' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:53.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:53 smithi102 ceph-mon[29849]: from='client.47048 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:53.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:53 smithi102 ceph-mon[41371]: osdmap e522: 8 total, 8 up, 8 in 2024-09-11T08:12:53.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:53 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3279590380' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:53.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:53 smithi102 ceph-mon[41371]: from='client.47048 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:53.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:53 smithi146 ceph-mon[33158]: osdmap e522: 8 total, 8 up, 8 in 2024-09-11T08:12:53.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:53 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3279590380' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:53.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:53 smithi146 ceph-mon[33158]: from='client.47048 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:54.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:54 smithi102 ceph-mon[29849]: pgmap v813: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:54.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:54 smithi102 ceph-mon[29849]: from='client.47048 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:12:54.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:54 smithi102 ceph-mon[29849]: osdmap e523: 8 total, 8 up, 8 in 2024-09-11T08:12:54.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:54 smithi102 ceph-mon[29849]: osdmap e524: 8 total, 8 up, 8 in 2024-09-11T08:12:54.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:54 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3279590380' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:54.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:54 smithi102 ceph-mon[29849]: from='client.47048 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:54.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:54 smithi102 ceph-mon[41371]: pgmap v813: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:54.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:54 smithi102 ceph-mon[41371]: from='client.47048 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:12:54.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:54 smithi102 ceph-mon[41371]: osdmap e523: 8 total, 8 up, 8 in 2024-09-11T08:12:54.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:54 smithi102 ceph-mon[41371]: osdmap e524: 8 total, 8 up, 8 in 2024-09-11T08:12:54.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:54 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3279590380' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:54.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:54 smithi102 ceph-mon[41371]: from='client.47048 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:54.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:54 smithi146 ceph-mon[33158]: pgmap v813: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:12:54.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:54 smithi146 ceph-mon[33158]: from='client.47048 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:12:54.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:54 smithi146 ceph-mon[33158]: osdmap e523: 8 total, 8 up, 8 in 2024-09-11T08:12:54.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:54 smithi146 ceph-mon[33158]: osdmap e524: 8 total, 8 up, 8 in 2024-09-11T08:12:54.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:54 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3279590380' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:54.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:54 smithi146 ceph-mon[33158]: from='client.47048 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:12:54.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:12:54 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:12:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:12:56.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:55 smithi102 ceph-mon[41371]: pgmap v816: 236 pgs: 64 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:56.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:55 smithi102 ceph-mon[41371]: from='client.47048 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:12:56.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:55 smithi102 ceph-mon[41371]: osdmap e525: 8 total, 8 up, 8 in 2024-09-11T08:12:56.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3279590380' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-107", "tierpool": "test-rados-api-smithi102-80667-107-cache"}]: dispatch 2024-09-11T08:12:56.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:55 smithi102 ceph-mon[41371]: from='client.47048 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-107", "tierpool": "test-rados-api-smithi102-80667-107-cache"}]: dispatch 2024-09-11T08:12:56.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:55 smithi102 ceph-mon[29849]: pgmap v816: 236 pgs: 64 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:56.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:55 smithi102 ceph-mon[29849]: from='client.47048 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:12:56.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:55 smithi102 ceph-mon[29849]: osdmap e525: 8 total, 8 up, 8 in 2024-09-11T08:12:56.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3279590380' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-107", "tierpool": "test-rados-api-smithi102-80667-107-cache"}]: dispatch 2024-09-11T08:12:56.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:55 smithi102 ceph-mon[29849]: from='client.47048 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-107", "tierpool": "test-rados-api-smithi102-80667-107-cache"}]: dispatch 2024-09-11T08:12:56.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:55 smithi146 ceph-mon[33158]: pgmap v816: 236 pgs: 64 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:12:56.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:55 smithi146 ceph-mon[33158]: from='client.47048 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:12:56.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:55 smithi146 ceph-mon[33158]: osdmap e525: 8 total, 8 up, 8 in 2024-09-11T08:12:56.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3279590380' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-107", "tierpool": "test-rados-api-smithi102-80667-107-cache"}]: dispatch 2024-09-11T08:12:56.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:55 smithi146 ceph-mon[33158]: from='client.47048 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-107", "tierpool": "test-rados-api-smithi102-80667-107-cache"}]: dispatch 2024-09-11T08:12:57.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:56 smithi102 ceph-mon[41371]: from='client.47048 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-107", "tierpool": "test-rados-api-smithi102-80667-107-cache"}]': finished 2024-09-11T08:12:57.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:56 smithi102 ceph-mon[41371]: osdmap e526: 8 total, 8 up, 8 in 2024-09-11T08:12:57.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:56 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3279590380' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-107", "overlaypool": "test-rados-api-smithi102-80667-107-cache"}]: dispatch 2024-09-11T08:12:57.149 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:56 smithi102 ceph-mon[41371]: from='client.47048 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-107", "overlaypool": "test-rados-api-smithi102-80667-107-cache"}]: dispatch 2024-09-11T08:12:57.149 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:56 smithi102 ceph-mon[29849]: from='client.47048 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-107", "tierpool": "test-rados-api-smithi102-80667-107-cache"}]': finished 2024-09-11T08:12:57.149 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:56 smithi102 ceph-mon[29849]: osdmap e526: 8 total, 8 up, 8 in 2024-09-11T08:12:57.149 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:56 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3279590380' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-107", "overlaypool": "test-rados-api-smithi102-80667-107-cache"}]: dispatch 2024-09-11T08:12:57.149 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:56 smithi102 ceph-mon[29849]: from='client.47048 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-107", "overlaypool": "test-rados-api-smithi102-80667-107-cache"}]: dispatch 2024-09-11T08:12:57.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:56 smithi146 ceph-mon[33158]: from='client.47048 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-107", "tierpool": "test-rados-api-smithi102-80667-107-cache"}]': finished 2024-09-11T08:12:57.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:56 smithi146 ceph-mon[33158]: osdmap e526: 8 total, 8 up, 8 in 2024-09-11T08:12:57.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:56 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3279590380' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-107", "overlaypool": "test-rados-api-smithi102-80667-107-cache"}]: dispatch 2024-09-11T08:12:57.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:56 smithi146 ceph-mon[33158]: from='client.47048 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-107", "overlaypool": "test-rados-api-smithi102-80667-107-cache"}]: dispatch 2024-09-11T08:12:58.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:57 smithi102 ceph-mon[29849]: pgmap v819: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:12:58.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:57 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:12:58.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:57 smithi102 ceph-mon[29849]: from='client.47048 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-107", "overlaypool": "test-rados-api-smithi102-80667-107-cache"}]': finished 2024-09-11T08:12:58.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:57 smithi102 ceph-mon[29849]: osdmap e527: 8 total, 8 up, 8 in 2024-09-11T08:12:58.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:57 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3279590380' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-107-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:12:58.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:57 smithi102 ceph-mon[29849]: from='client.47048 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-107-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:12:58.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:57 smithi102 ceph-mon[41371]: pgmap v819: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:12:58.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:57 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:12:58.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:57 smithi102 ceph-mon[41371]: from='client.47048 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-107", "overlaypool": "test-rados-api-smithi102-80667-107-cache"}]': finished 2024-09-11T08:12:58.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:57 smithi102 ceph-mon[41371]: osdmap e527: 8 total, 8 up, 8 in 2024-09-11T08:12:58.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:57 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3279590380' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-107-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:12:58.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:57 smithi102 ceph-mon[41371]: from='client.47048 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-107-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:12:58.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:57 smithi146 ceph-mon[33158]: pgmap v819: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:12:58.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:57 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:12:58.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:57 smithi146 ceph-mon[33158]: from='client.47048 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-107", "overlaypool": "test-rados-api-smithi102-80667-107-cache"}]': finished 2024-09-11T08:12:58.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:57 smithi146 ceph-mon[33158]: osdmap e527: 8 total, 8 up, 8 in 2024-09-11T08:12:58.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:57 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3279590380' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-107-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:12:58.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:57 smithi146 ceph-mon[33158]: from='client.47048 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-107-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:12:59.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:58 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:12:59.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:58 smithi102 ceph-mon[41371]: from='client.47048 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-107-cache", "mode": "writeback"}]': finished 2024-09-11T08:12:59.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:58 smithi102 ceph-mon[41371]: osdmap e528: 8 total, 8 up, 8 in 2024-09-11T08:12:59.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:58 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3279590380' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-107"}]: dispatch 2024-09-11T08:12:59.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:58 smithi102 ceph-mon[41371]: from='client.47048 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-107"}]: dispatch 2024-09-11T08:12:59.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:58 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:12:59.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:58 smithi102 ceph-mon[29849]: from='client.47048 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-107-cache", "mode": "writeback"}]': finished 2024-09-11T08:12:59.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:58 smithi102 ceph-mon[29849]: osdmap e528: 8 total, 8 up, 8 in 2024-09-11T08:12:59.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:58 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3279590380' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-107"}]: dispatch 2024-09-11T08:12:59.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:58 smithi102 ceph-mon[29849]: from='client.47048 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-107"}]: dispatch 2024-09-11T08:12:59.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:58 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:12:59.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:58 smithi146 ceph-mon[33158]: from='client.47048 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-107-cache", "mode": "writeback"}]': finished 2024-09-11T08:12:59.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:58 smithi146 ceph-mon[33158]: osdmap e528: 8 total, 8 up, 8 in 2024-09-11T08:12:59.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:58 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3279590380' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-107"}]: dispatch 2024-09-11T08:12:59.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:58 smithi146 ceph-mon[33158]: from='client.47048 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-107"}]: dispatch 2024-09-11T08:13:00.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:12:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:12:59] "GET /metrics HTTP/1.1" 200 231086 "" "Prometheus/2.43.0" 2024-09-11T08:13:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:59 smithi102 ceph-mon[29849]: pgmap v822: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:13:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:59 smithi102 ceph-mon[29849]: from='client.47048 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-107"}]': finished 2024-09-11T08:13:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:59 smithi102 ceph-mon[29849]: osdmap e529: 8 total, 8 up, 8 in 2024-09-11T08:13:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:59 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3279590380' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-107", "tierpool": "test-rados-api-smithi102-80667-107-cache"}]: dispatch 2024-09-11T08:13:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:59 smithi102 ceph-mon[29849]: from='client.47048 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-107", "tierpool": "test-rados-api-smithi102-80667-107-cache"}]: dispatch 2024-09-11T08:13:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:59 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:13:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:12:59 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:00.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:59 smithi102 ceph-mon[41371]: pgmap v822: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:13:00.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:59 smithi102 ceph-mon[41371]: from='client.47048 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-107"}]': finished 2024-09-11T08:13:00.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:59 smithi102 ceph-mon[41371]: osdmap e529: 8 total, 8 up, 8 in 2024-09-11T08:13:00.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:59 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3279590380' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-107", "tierpool": "test-rados-api-smithi102-80667-107-cache"}]: dispatch 2024-09-11T08:13:00.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:59 smithi102 ceph-mon[41371]: from='client.47048 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-107", "tierpool": "test-rados-api-smithi102-80667-107-cache"}]: dispatch 2024-09-11T08:13:00.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:59 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:13:00.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:12:59 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:00.183 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:12:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:13:00.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:59 smithi146 ceph-mon[33158]: pgmap v822: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:13:00.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:59 smithi146 ceph-mon[33158]: from='client.47048 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-107"}]': finished 2024-09-11T08:13:00.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:59 smithi146 ceph-mon[33158]: osdmap e529: 8 total, 8 up, 8 in 2024-09-11T08:13:00.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:59 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3279590380' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-107", "tierpool": "test-rados-api-smithi102-80667-107-cache"}]: dispatch 2024-09-11T08:13:00.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:59 smithi146 ceph-mon[33158]: from='client.47048 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-107", "tierpool": "test-rados-api-smithi102-80667-107-cache"}]: dispatch 2024-09-11T08:13:00.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:59 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:13:00.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:12:59 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:01.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:00 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:13:01.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:00 smithi102 ceph-mon[41371]: from='client.47048 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-107", "tierpool": "test-rados-api-smithi102-80667-107-cache"}]': finished 2024-09-11T08:13:01.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:00 smithi102 ceph-mon[41371]: osdmap e530: 8 total, 8 up, 8 in 2024-09-11T08:13:01.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:00 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:01.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:00 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:13:01.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:00 smithi102 ceph-mon[29849]: from='client.47048 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-107", "tierpool": "test-rados-api-smithi102-80667-107-cache"}]': finished 2024-09-11T08:13:01.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:00 smithi102 ceph-mon[29849]: osdmap e530: 8 total, 8 up, 8 in 2024-09-11T08:13:01.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:00 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:01.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:00 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:13:01.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:00 smithi146 ceph-mon[33158]: from='client.47048 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-107", "tierpool": "test-rados-api-smithi102-80667-107-cache"}]': finished 2024-09-11T08:13:01.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:00 smithi146 ceph-mon[33158]: osdmap e530: 8 total, 8 up, 8 in 2024-09-11T08:13:01.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:00 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:02.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:01 smithi102 ceph-mon[41371]: pgmap v825: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:13:02.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:01 smithi102 ceph-mon[41371]: osdmap e531: 8 total, 8 up, 8 in 2024-09-11T08:13:02.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:01 smithi102 ceph-mon[29849]: pgmap v825: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:13:02.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:01 smithi102 ceph-mon[29849]: osdmap e531: 8 total, 8 up, 8 in 2024-09-11T08:13:02.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:01 smithi146 ceph-mon[33158]: pgmap v825: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:13:02.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:01 smithi146 ceph-mon[33158]: osdmap e531: 8 total, 8 up, 8 in 2024-09-11T08:13:03.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:02 smithi102 ceph-mon[29849]: osdmap e532: 8 total, 8 up, 8 in 2024-09-11T08:13:03.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:02 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80667-109"}]: dispatch 2024-09-11T08:13:03.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:02 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80667-109"}]: dispatch 2024-09-11T08:13:03.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:02 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi102-80667-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:13:03.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:02 smithi102 ceph-mon[41371]: osdmap e532: 8 total, 8 up, 8 in 2024-09-11T08:13:03.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:02 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80667-109"}]: dispatch 2024-09-11T08:13:03.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:02 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80667-109"}]: dispatch 2024-09-11T08:13:03.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:02 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi102-80667-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:13:03.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:02 smithi146 ceph-mon[33158]: osdmap e532: 8 total, 8 up, 8 in 2024-09-11T08:13:03.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:02 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80667-109"}]: dispatch 2024-09-11T08:13:03.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:02 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80667-109"}]: dispatch 2024-09-11T08:13:03.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:02 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi102-80667-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:13:04.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:04 smithi102 ceph-mon[29849]: pgmap v828: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:04.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:04 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi102-80667-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:13:04.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:04 smithi102 ceph-mon[29849]: osdmap e533: 8 total, 8 up, 8 in 2024-09-11T08:13:04.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:04 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi102-80667-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi102-80667-109"}]: dispatch 2024-09-11T08:13:04.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:04 smithi102 ceph-mon[41371]: pgmap v828: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:04.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:04 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi102-80667-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:13:04.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:04 smithi102 ceph-mon[41371]: osdmap e533: 8 total, 8 up, 8 in 2024-09-11T08:13:04.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:04 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi102-80667-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi102-80667-109"}]: dispatch 2024-09-11T08:13:04.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:04 smithi146 ceph-mon[33158]: pgmap v828: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:04.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:04 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi102-80667-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:13:04.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:04 smithi146 ceph-mon[33158]: osdmap e533: 8 total, 8 up, 8 in 2024-09-11T08:13:04.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:04 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi102-80667-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi102-80667-109"}]: dispatch 2024-09-11T08:13:04.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:13:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:13:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:13:05.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:05 smithi102 ceph-mon[29849]: osdmap e534: 8 total, 8 up, 8 in 2024-09-11T08:13:05.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:05 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi102-80667-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi102-80667-109"}]': finished 2024-09-11T08:13:05.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:05 smithi102 ceph-mon[29849]: osdmap e535: 8 total, 8 up, 8 in 2024-09-11T08:13:05.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:05 smithi102 ceph-mon[41371]: osdmap e534: 8 total, 8 up, 8 in 2024-09-11T08:13:05.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:05 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi102-80667-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi102-80667-109"}]': finished 2024-09-11T08:13:05.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:05 smithi102 ceph-mon[41371]: osdmap e535: 8 total, 8 up, 8 in 2024-09-11T08:13:05.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:05 smithi146 ceph-mon[33158]: osdmap e534: 8 total, 8 up, 8 in 2024-09-11T08:13:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:05 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi102-80667-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi102-80667-109"}]': finished 2024-09-11T08:13:05.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:05 smithi146 ceph-mon[33158]: osdmap e535: 8 total, 8 up, 8 in 2024-09-11T08:13:06.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:06 smithi102 ceph-mon[29849]: pgmap v831: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:13:06.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:06 smithi102 ceph-mon[29849]: osdmap e536: 8 total, 8 up, 8 in 2024-09-11T08:13:06.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:06 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:13:06.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:06 smithi102 ceph-mon[41371]: pgmap v831: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:13:06.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:06 smithi102 ceph-mon[41371]: osdmap e536: 8 total, 8 up, 8 in 2024-09-11T08:13:06.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:06 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:13:06.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:06 smithi146 ceph-mon[33158]: pgmap v831: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:13:06.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:06 smithi146 ceph-mon[33158]: osdmap e536: 8 total, 8 up, 8 in 2024-09-11T08:13:06.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:06 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:13:08.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:07 smithi102 ceph-mon[29849]: pgmap v834: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:08.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:13:08.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:07 smithi102 ceph-mon[29849]: osdmap e537: 8 total, 8 up, 8 in 2024-09-11T08:13:08.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-109", "tierpool": "test-rados-api-smithi102-80667-109-cache"}]: dispatch 2024-09-11T08:13:08.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:07 smithi102 ceph-mon[29849]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:13:08.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:07 smithi102 ceph-mon[41371]: pgmap v834: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:08.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:13:08.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:07 smithi102 ceph-mon[41371]: osdmap e537: 8 total, 8 up, 8 in 2024-09-11T08:13:08.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-109", "tierpool": "test-rados-api-smithi102-80667-109-cache"}]: dispatch 2024-09-11T08:13:08.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:07 smithi102 ceph-mon[41371]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:13:08.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:07 smithi146 ceph-mon[33158]: pgmap v834: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:08.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:13:08.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:07 smithi146 ceph-mon[33158]: osdmap e537: 8 total, 8 up, 8 in 2024-09-11T08:13:08.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-109", "tierpool": "test-rados-api-smithi102-80667-109-cache"}]: dispatch 2024-09-11T08:13:08.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:07 smithi146 ceph-mon[33158]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:13:09.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:08 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-109", "tierpool": "test-rados-api-smithi102-80667-109-cache"}]': finished 2024-09-11T08:13:09.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:08 smithi102 ceph-mon[41371]: osdmap e538: 8 total, 8 up, 8 in 2024-09-11T08:13:09.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:08 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-109", "overlaypool": "test-rados-api-smithi102-80667-109-cache"}]: dispatch 2024-09-11T08:13:09.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:08 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-109", "tierpool": "test-rados-api-smithi102-80667-109-cache"}]': finished 2024-09-11T08:13:09.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:08 smithi102 ceph-mon[29849]: osdmap e538: 8 total, 8 up, 8 in 2024-09-11T08:13:09.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:08 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-109", "overlaypool": "test-rados-api-smithi102-80667-109-cache"}]: dispatch 2024-09-11T08:13:09.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:08 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-109", "tierpool": "test-rados-api-smithi102-80667-109-cache"}]': finished 2024-09-11T08:13:09.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:08 smithi146 ceph-mon[33158]: osdmap e538: 8 total, 8 up, 8 in 2024-09-11T08:13:09.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:08 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-109", "overlaypool": "test-rados-api-smithi102-80667-109-cache"}]: dispatch 2024-09-11T08:13:10.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:13:09 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:13:09] "GET /metrics HTTP/1.1" 200 231086 "" "Prometheus/2.43.0" 2024-09-11T08:13:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:09 smithi102 ceph-mon[29849]: pgmap v837: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:09 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-109", "overlaypool": "test-rados-api-smithi102-80667-109-cache"}]': finished 2024-09-11T08:13:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:09 smithi102 ceph-mon[29849]: osdmap e539: 8 total, 8 up, 8 in 2024-09-11T08:13:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:09 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-109-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:13:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:09 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:13:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:09 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-109-cache", "mode": "writeback"}]': finished 2024-09-11T08:13:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:09 smithi102 ceph-mon[29849]: osdmap e540: 8 total, 8 up, 8 in 2024-09-11T08:13:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:09 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:13:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:09 smithi102 ceph-mon[41371]: pgmap v837: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:09 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-109", "overlaypool": "test-rados-api-smithi102-80667-109-cache"}]': finished 2024-09-11T08:13:10.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:09 smithi102 ceph-mon[41371]: osdmap e539: 8 total, 8 up, 8 in 2024-09-11T08:13:10.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:09 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-109-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:13:10.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:09 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:13:10.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:09 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-109-cache", "mode": "writeback"}]': finished 2024-09-11T08:13:10.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:09 smithi102 ceph-mon[41371]: osdmap e540: 8 total, 8 up, 8 in 2024-09-11T08:13:10.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:09 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:13:10.183 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:13:09 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:13:10.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:09 smithi146 ceph-mon[33158]: pgmap v837: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:10.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:09 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-109", "overlaypool": "test-rados-api-smithi102-80667-109-cache"}]': finished 2024-09-11T08:13:10.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:09 smithi146 ceph-mon[33158]: osdmap e539: 8 total, 8 up, 8 in 2024-09-11T08:13:10.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:09 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-109-cache", "mode": "writeback"}]: dispatch 2024-09-11T08:13:10.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:09 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:13:10.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:09 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-109-cache", "mode": "writeback"}]': finished 2024-09-11T08:13:10.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:09 smithi146 ceph-mon[33158]: osdmap e540: 8 total, 8 up, 8 in 2024-09-11T08:13:10.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:09 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:13:11.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:10 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:11.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:13:11.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:10 smithi102 ceph-mon[41371]: osdmap e541: 8 total, 8 up, 8 in 2024-09-11T08:13:11.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:13:11.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:10 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:11.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:13:11.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:10 smithi102 ceph-mon[29849]: osdmap e541: 8 total, 8 up, 8 in 2024-09-11T08:13:11.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:13:11.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:10 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:11.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:13:11.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:10 smithi146 ceph-mon[33158]: osdmap e541: 8 total, 8 up, 8 in 2024-09-11T08:13:11.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:13:12.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:11 smithi102 ceph-mon[41371]: pgmap v840: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:12.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:11 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:13:12.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:11 smithi102 ceph-mon[41371]: osdmap e542: 8 total, 8 up, 8 in 2024-09-11T08:13:12.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:11 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:13:12.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:11 smithi102 ceph-mon[29849]: pgmap v840: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:12.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:11 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:13:12.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:11 smithi102 ceph-mon[29849]: osdmap e542: 8 total, 8 up, 8 in 2024-09-11T08:13:12.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:11 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:13:12.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:11 smithi146 ceph-mon[33158]: pgmap v840: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:12.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:11 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:13:12.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:11 smithi146 ceph-mon[33158]: osdmap e542: 8 total, 8 up, 8 in 2024-09-11T08:13:12.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:11 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:13:13.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:12 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:13:13.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:13:13.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:12 smithi102 ceph-mon[41371]: osdmap e543: 8 total, 8 up, 8 in 2024-09-11T08:13:13.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-09-11T08:13:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:12 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:13:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:13:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:12 smithi102 ceph-mon[29849]: osdmap e543: 8 total, 8 up, 8 in 2024-09-11T08:13:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-09-11T08:13:13.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:12 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:13:13.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:13:13.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:12 smithi146 ceph-mon[33158]: osdmap e543: 8 total, 8 up, 8 in 2024-09-11T08:13:13.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-09-11T08:13:14.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:13 smithi102 ceph-mon[41371]: pgmap v843: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:14.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-11T08:13:14.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:13 smithi102 ceph-mon[41371]: osdmap e544: 8 total, 8 up, 8 in 2024-09-11T08:13:14.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:13 smithi102 ceph-mon[29849]: pgmap v843: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:14.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-11T08:13:14.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:13 smithi102 ceph-mon[29849]: osdmap e544: 8 total, 8 up, 8 in 2024-09-11T08:13:14.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:13 smithi146 ceph-mon[33158]: pgmap v843: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:14.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-11T08:13:14.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:13 smithi146 ceph-mon[33158]: osdmap e544: 8 total, 8 up, 8 in 2024-09-11T08:13:14.888 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:13:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:13:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:13:15.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-109"}]: dispatch 2024-09-11T08:13:15.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:14 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:13:15.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:14 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:15.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-109"}]: dispatch 2024-09-11T08:13:15.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:14 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:13:15.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:14 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:15.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-109"}]: dispatch 2024-09-11T08:13:15.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:14 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:13:15.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:14 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:16.174 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:15 smithi146 ceph-mon[33158]: pgmap v846: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:13:16.174 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:15 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-109"}]': finished 2024-09-11T08:13:16.174 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:15 smithi146 ceph-mon[33158]: osdmap e545: 8 total, 8 up, 8 in 2024-09-11T08:13:16.174 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:15 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-109", "tierpool": "test-rados-api-smithi102-80667-109-cache"}]: dispatch 2024-09-11T08:13:16.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:15 smithi102 ceph-mon[29849]: pgmap v846: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:13:16.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:15 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-109"}]': finished 2024-09-11T08:13:16.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:15 smithi102 ceph-mon[29849]: osdmap e545: 8 total, 8 up, 8 in 2024-09-11T08:13:16.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:15 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-109", "tierpool": "test-rados-api-smithi102-80667-109-cache"}]: dispatch 2024-09-11T08:13:16.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:15 smithi102 ceph-mon[41371]: pgmap v846: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:13:16.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:15 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-109"}]': finished 2024-09-11T08:13:16.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:15 smithi102 ceph-mon[41371]: osdmap e545: 8 total, 8 up, 8 in 2024-09-11T08:13:16.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:15 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-109", "tierpool": "test-rados-api-smithi102-80667-109-cache"}]: dispatch 2024-09-11T08:13:17.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-109", "tierpool": "test-rados-api-smithi102-80667-109-cache"}]': finished 2024-09-11T08:13:17.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:16 smithi146 ceph-mon[33158]: osdmap e546: 8 total, 8 up, 8 in 2024-09-11T08:13:17.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-109", "tierpool": "test-rados-api-smithi102-80667-109-cache"}]': finished 2024-09-11T08:13:17.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:16 smithi102 ceph-mon[29849]: osdmap e546: 8 total, 8 up, 8 in 2024-09-11T08:13:17.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-109", "tierpool": "test-rados-api-smithi102-80667-109-cache"}]': finished 2024-09-11T08:13:17.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:16 smithi102 ceph-mon[41371]: osdmap e546: 8 total, 8 up, 8 in 2024-09-11T08:13:18.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:17 smithi146 ceph-mon[33158]: pgmap v849: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:13:18.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:17 smithi146 ceph-mon[33158]: osdmap e547: 8 total, 8 up, 8 in 2024-09-11T08:13:18.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:17 smithi146 ceph-mon[33158]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:13:18.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:17 smithi146 ceph-mon[33158]: osdmap e548: 8 total, 8 up, 8 in 2024-09-11T08:13:18.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80667-109"}]: dispatch 2024-09-11T08:13:18.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:17 smithi102 ceph-mon[29849]: pgmap v849: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:13:18.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:17 smithi102 ceph-mon[29849]: osdmap e547: 8 total, 8 up, 8 in 2024-09-11T08:13:18.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:17 smithi102 ceph-mon[29849]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:13:18.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:17 smithi102 ceph-mon[29849]: osdmap e548: 8 total, 8 up, 8 in 2024-09-11T08:13:18.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80667-109"}]: dispatch 2024-09-11T08:13:18.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:17 smithi102 ceph-mon[41371]: pgmap v849: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:13:18.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:17 smithi102 ceph-mon[41371]: osdmap e547: 8 total, 8 up, 8 in 2024-09-11T08:13:18.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:17 smithi102 ceph-mon[41371]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:13:18.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:17 smithi102 ceph-mon[41371]: osdmap e548: 8 total, 8 up, 8 in 2024-09-11T08:13:18.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80667-109"}]: dispatch 2024-09-11T08:13:20.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:13:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:13:19] "GET /metrics HTTP/1.1" 200 231058 "" "Prometheus/2.43.0" 2024-09-11T08:13:20.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:19 smithi102 ceph-mon[41371]: pgmap v852: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:20.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80667-109"}]': finished 2024-09-11T08:13:20.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:19 smithi102 ceph-mon[41371]: osdmap e549: 8 total, 8 up, 8 in 2024-09-11T08:13:20.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80667-109"}]: dispatch 2024-09-11T08:13:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:19 smithi102 ceph-mon[29849]: pgmap v852: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80667-109"}]': finished 2024-09-11T08:13:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:19 smithi102 ceph-mon[29849]: osdmap e549: 8 total, 8 up, 8 in 2024-09-11T08:13:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80667-109"}]: dispatch 2024-09-11T08:13:20.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:19 smithi146 ceph-mon[33158]: pgmap v852: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:20.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80667-109"}]': finished 2024-09-11T08:13:20.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:19 smithi146 ceph-mon[33158]: osdmap e549: 8 total, 8 up, 8 in 2024-09-11T08:13:20.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80667-109"}]: dispatch 2024-09-11T08:13:20.184 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:13:19 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:13:21.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80667-109"}]': finished 2024-09-11T08:13:21.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:20 smithi146 ceph-mon[33158]: osdmap e550: 8 total, 8 up, 8 in 2024-09-11T08:13:21.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80667-109"}]': finished 2024-09-11T08:13:21.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:20 smithi102 ceph-mon[41371]: osdmap e550: 8 total, 8 up, 8 in 2024-09-11T08:13:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2209339823' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80667-109"}]': finished 2024-09-11T08:13:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:20 smithi102 ceph-mon[29849]: osdmap e550: 8 total, 8 up, 8 in 2024-09-11T08:13:22.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:21 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:22.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:21 smithi146 ceph-mon[33158]: pgmap v855: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-11T08:13:22.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:21 smithi146 ceph-mon[33158]: osdmap e551: 8 total, 8 up, 8 in 2024-09-11T08:13:22.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:21 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi102-80667-104"}]: dispatch 2024-09-11T08:13:22.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:21 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:22.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:21 smithi102 ceph-mon[29849]: pgmap v855: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-11T08:13:22.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:21 smithi102 ceph-mon[29849]: osdmap e551: 8 total, 8 up, 8 in 2024-09-11T08:13:22.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:21 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi102-80667-104"}]: dispatch 2024-09-11T08:13:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:21 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:21 smithi102 ceph-mon[41371]: pgmap v855: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-11T08:13:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:21 smithi102 ceph-mon[41371]: osdmap e551: 8 total, 8 up, 8 in 2024-09-11T08:13:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:21 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi102-80667-104"}]: dispatch 2024-09-11T08:13:23.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi102-80667-104"}]': finished 2024-09-11T08:13:23.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:22 smithi146 ceph-mon[33158]: osdmap e552: 8 total, 8 up, 8 in 2024-09-11T08:13:23.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi102-80667-104"}]: dispatch 2024-09-11T08:13:23.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:22 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:13:23.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi102-80667-104"}]': finished 2024-09-11T08:13:23.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:22 smithi102 ceph-mon[41371]: osdmap e552: 8 total, 8 up, 8 in 2024-09-11T08:13:23.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi102-80667-104"}]: dispatch 2024-09-11T08:13:23.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:22 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:13:23.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi102-80667-104"}]': finished 2024-09-11T08:13:23.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:22 smithi102 ceph-mon[29849]: osdmap e552: 8 total, 8 up, 8 in 2024-09-11T08:13:23.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi102-80667-104"}]: dispatch 2024-09-11T08:13:23.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:22 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:13:24.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:23 smithi146 ceph-mon[33158]: pgmap v858: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-11T08:13:24.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi102-80667-104"}]': finished 2024-09-11T08:13:24.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:23 smithi146 ceph-mon[33158]: osdmap e553: 8 total, 8 up, 8 in 2024-09-11T08:13:24.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:13:24.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:13:24.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi102-80667-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:13:24.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:23 smithi102 ceph-mon[29849]: pgmap v858: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-11T08:13:24.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi102-80667-104"}]': finished 2024-09-11T08:13:24.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:23 smithi102 ceph-mon[29849]: osdmap e553: 8 total, 8 up, 8 in 2024-09-11T08:13:24.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:13:24.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:13:24.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi102-80667-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:13:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:23 smithi102 ceph-mon[41371]: pgmap v858: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-11T08:13:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2017796322' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi102-80667-104"}]': finished 2024-09-11T08:13:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:23 smithi102 ceph-mon[41371]: osdmap e553: 8 total, 8 up, 8 in 2024-09-11T08:13:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:13:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:13:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi102-80667-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T08:13:24.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:13:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:13:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:13:25.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi102-80667-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:13:25.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:24 smithi102 ceph-mon[29849]: osdmap e554: 8 total, 8 up, 8 in 2024-09-11T08:13:25.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi102-80667-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:13:25.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi102-80667-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:13:25.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:24 smithi102 ceph-mon[41371]: osdmap e554: 8 total, 8 up, 8 in 2024-09-11T08:13:25.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi102-80667-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:13:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi102-80667-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T08:13:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:24 smithi146 ceph-mon[33158]: osdmap e554: 8 total, 8 up, 8 in 2024-09-11T08:13:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi102-80667-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:13:26.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:25 smithi102 ceph-mon[41371]: pgmap v861: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:13:26.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:25 smithi102 ceph-mon[41371]: osdmap e555: 8 total, 8 up, 8 in 2024-09-11T08:13:26.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:25 smithi102 ceph-mon[29849]: pgmap v861: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:13:26.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:25 smithi102 ceph-mon[29849]: osdmap e555: 8 total, 8 up, 8 in 2024-09-11T08:13:26.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:25 smithi146 ceph-mon[33158]: pgmap v861: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:13:26.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:25 smithi146 ceph-mon[33158]: osdmap e555: 8 total, 8 up, 8 in 2024-09-11T08:13:27.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi102-80667-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:13:27.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:26 smithi102 ceph-mon[29849]: osdmap e556: 8 total, 8 up, 8 in 2024-09-11T08:13:27.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi102-80667-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:13:27.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:26 smithi102 ceph-mon[41371]: osdmap e556: 8 total, 8 up, 8 in 2024-09-11T08:13:27.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi102-80667-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:13:27.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:26 smithi146 ceph-mon[33158]: osdmap e556: 8 total, 8 up, 8 in 2024-09-11T08:13:28.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:27 smithi102 ceph-mon[29849]: pgmap v864: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:28.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:27 smithi102 ceph-mon[29849]: osdmap e557: 8 total, 8 up, 8 in 2024-09-11T08:13:28.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:13:28.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:27 smithi102 ceph-mon[41371]: pgmap v864: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:28.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:27 smithi102 ceph-mon[41371]: osdmap e557: 8 total, 8 up, 8 in 2024-09-11T08:13:28.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:13:28.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:27 smithi146 ceph-mon[33158]: pgmap v864: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:28.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:27 smithi146 ceph-mon[33158]: osdmap e557: 8 total, 8 up, 8 in 2024-09-11T08:13:28.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:13:29.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:13:29.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:28 smithi102 ceph-mon[41371]: osdmap e558: 8 total, 8 up, 8 in 2024-09-11T08:13:29.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:13:29.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:13:29.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:28 smithi102 ceph-mon[29849]: osdmap e558: 8 total, 8 up, 8 in 2024-09-11T08:13:29.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:13:29.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:13:29.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:28 smithi146 ceph-mon[33158]: osdmap e558: 8 total, 8 up, 8 in 2024-09-11T08:13:29.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:13:30.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:29 smithi102 ceph-mon[29849]: pgmap v867: 204 pgs: 40 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:30.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:13:30.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:29 smithi102 ceph-mon[29849]: osdmap e559: 8 total, 8 up, 8 in 2024-09-11T08:13:30.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-112"}]: dispatch 2024-09-11T08:13:30.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:29 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:30.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:13:29 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:13:29] "GET /metrics HTTP/1.1" 200 228656 "" "Prometheus/2.43.0" 2024-09-11T08:13:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:29 smithi102 ceph-mon[41371]: pgmap v867: 204 pgs: 40 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:13:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:29 smithi102 ceph-mon[41371]: osdmap e559: 8 total, 8 up, 8 in 2024-09-11T08:13:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-112"}]: dispatch 2024-09-11T08:13:30.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:29 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:30.433 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:13:29 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:13:30.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:29 smithi146 ceph-mon[33158]: pgmap v867: 204 pgs: 40 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:30.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:13:30.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:29 smithi146 ceph-mon[33158]: osdmap e559: 8 total, 8 up, 8 in 2024-09-11T08:13:30.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-112"}]: dispatch 2024-09-11T08:13:30.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:29 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:31.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:30 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-112"}]': finished 2024-09-11T08:13:31.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:30 smithi102 ceph-mon[41371]: osdmap e560: 8 total, 8 up, 8 in 2024-09-11T08:13:31.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:30 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:13:31.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:30 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-112"}]': finished 2024-09-11T08:13:31.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:30 smithi102 ceph-mon[29849]: osdmap e560: 8 total, 8 up, 8 in 2024-09-11T08:13:31.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:30 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:13:31.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:30 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-112"}]': finished 2024-09-11T08:13:31.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:30 smithi146 ceph-mon[33158]: osdmap e560: 8 total, 8 up, 8 in 2024-09-11T08:13:31.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:30 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:13:32.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:31 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:32.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:31 smithi102 ceph-mon[41371]: pgmap v870: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:13:32.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:31 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:13:32.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:31 smithi102 ceph-mon[41371]: osdmap e561: 8 total, 8 up, 8 in 2024-09-11T08:13:32.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:31 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-112"}]: dispatch 2024-09-11T08:13:32.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:31 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:32.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:31 smithi102 ceph-mon[29849]: pgmap v870: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:13:32.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:31 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:13:32.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:31 smithi102 ceph-mon[29849]: osdmap e561: 8 total, 8 up, 8 in 2024-09-11T08:13:32.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:31 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-112"}]: dispatch 2024-09-11T08:13:32.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:31 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:32.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:31 smithi146 ceph-mon[33158]: pgmap v870: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:13:32.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:31 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:13:32.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:31 smithi146 ceph-mon[33158]: osdmap e561: 8 total, 8 up, 8 in 2024-09-11T08:13:32.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:31 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-112"}]: dispatch 2024-09-11T08:13:33.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-112"}]': finished 2024-09-11T08:13:33.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:32 smithi102 ceph-mon[29849]: osdmap e562: 8 total, 8 up, 8 in 2024-09-11T08:13:33.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:32 smithi102 ceph-mon[29849]: pgmap v873: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:13:33.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-112"}]': finished 2024-09-11T08:13:33.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:32 smithi102 ceph-mon[41371]: osdmap e562: 8 total, 8 up, 8 in 2024-09-11T08:13:33.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:32 smithi102 ceph-mon[41371]: pgmap v873: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:13:33.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-112"}]': finished 2024-09-11T08:13:33.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:32 smithi146 ceph-mon[33158]: osdmap e562: 8 total, 8 up, 8 in 2024-09-11T08:13:33.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:32 smithi146 ceph-mon[33158]: pgmap v873: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:13:34.335 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:33 smithi146 ceph-mon[33158]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:13:34.335 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:33 smithi146 ceph-mon[33158]: osdmap e563: 8 total, 8 up, 8 in 2024-09-11T08:13:34.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:33 smithi102 ceph-mon[29849]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:13:34.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:33 smithi102 ceph-mon[29849]: osdmap e563: 8 total, 8 up, 8 in 2024-09-11T08:13:34.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:33 smithi102 ceph-mon[41371]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:13:34.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:33 smithi102 ceph-mon[41371]: osdmap e563: 8 total, 8 up, 8 in 2024-09-11T08:13:34.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:13:34 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:13:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:13:35.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:35 smithi102 ceph-mon[29849]: osdmap e564: 8 total, 8 up, 8 in 2024-09-11T08:13:35.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:13:35.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:35 smithi102 ceph-mon[29849]: pgmap v876: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:13:35.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:35 smithi102 ceph-mon[41371]: osdmap e564: 8 total, 8 up, 8 in 2024-09-11T08:13:35.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:13:35.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:35 smithi102 ceph-mon[41371]: pgmap v876: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:13:35.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:35 smithi146 ceph-mon[33158]: osdmap e564: 8 total, 8 up, 8 in 2024-09-11T08:13:35.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:13:35.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:35 smithi146 ceph-mon[33158]: pgmap v876: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:13:36.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:36 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:13:36.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:36 smithi102 ceph-mon[41371]: osdmap e565: 8 total, 8 up, 8 in 2024-09-11T08:13:36.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:36 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:13:36.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:36 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:13:36.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:36 smithi102 ceph-mon[29849]: osdmap e565: 8 total, 8 up, 8 in 2024-09-11T08:13:36.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:36 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:13:36.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:36 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:13:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:36 smithi146 ceph-mon[33158]: osdmap e565: 8 total, 8 up, 8 in 2024-09-11T08:13:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:36 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:13:37.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:13:37.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:37 smithi102 ceph-mon[29849]: osdmap e566: 8 total, 8 up, 8 in 2024-09-11T08:13:37.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-114"}]: dispatch 2024-09-11T08:13:37.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:37 smithi102 ceph-mon[29849]: pgmap v879: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-11T08:13:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:13:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:37 smithi102 ceph-mon[41371]: osdmap e566: 8 total, 8 up, 8 in 2024-09-11T08:13:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-114"}]: dispatch 2024-09-11T08:13:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:37 smithi102 ceph-mon[41371]: pgmap v879: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-11T08:13:37.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:13:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:37 smithi146 ceph-mon[33158]: osdmap e566: 8 total, 8 up, 8 in 2024-09-11T08:13:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-114"}]: dispatch 2024-09-11T08:13:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:37 smithi146 ceph-mon[33158]: pgmap v879: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-11T08:13:38.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-114"}]': finished 2024-09-11T08:13:38.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:38 smithi102 ceph-mon[29849]: osdmap e567: 8 total, 8 up, 8 in 2024-09-11T08:13:38.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-114", "mode": "writeback"}]: dispatch 2024-09-11T08:13:38.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-114"}]': finished 2024-09-11T08:13:38.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:38 smithi102 ceph-mon[41371]: osdmap e567: 8 total, 8 up, 8 in 2024-09-11T08:13:38.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-114", "mode": "writeback"}]: dispatch 2024-09-11T08:13:38.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-114"}]': finished 2024-09-11T08:13:38.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:38 smithi146 ceph-mon[33158]: osdmap e567: 8 total, 8 up, 8 in 2024-09-11T08:13:38.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-114", "mode": "writeback"}]: dispatch 2024-09-11T08:13:39.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:39 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:13:39.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:39 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-114", "mode": "writeback"}]': finished 2024-09-11T08:13:39.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:39 smithi102 ceph-mon[29849]: osdmap e568: 8 total, 8 up, 8 in 2024-09-11T08:13:39.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:39 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:13:39.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:39 smithi102 ceph-mon[29849]: pgmap v882: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:13:39.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:39 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:13:39.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:39 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-114", "mode": "writeback"}]': finished 2024-09-11T08:13:39.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:39 smithi102 ceph-mon[41371]: osdmap e568: 8 total, 8 up, 8 in 2024-09-11T08:13:39.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:39 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:13:39.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:39 smithi102 ceph-mon[41371]: pgmap v882: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:13:39.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:39 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:13:39.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:39 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-114", "mode": "writeback"}]': finished 2024-09-11T08:13:39.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:39 smithi146 ceph-mon[33158]: osdmap e568: 8 total, 8 up, 8 in 2024-09-11T08:13:39.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:39 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:13:39.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:39 smithi146 ceph-mon[33158]: pgmap v882: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:13:40.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:13:40.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:40 smithi102 ceph-mon[29849]: osdmap e569: 8 total, 8 up, 8 in 2024-09-11T08:13:40.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-114"}]: dispatch 2024-09-11T08:13:40.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:13:39 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:13:39] "GET /metrics HTTP/1.1" 200 228656 "" "Prometheus/2.43.0" 2024-09-11T08:13:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:13:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:40 smithi102 ceph-mon[41371]: osdmap e569: 8 total, 8 up, 8 in 2024-09-11T08:13:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-114"}]: dispatch 2024-09-11T08:13:40.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:13:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:40 smithi146 ceph-mon[33158]: osdmap e569: 8 total, 8 up, 8 in 2024-09-11T08:13:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-114"}]: dispatch 2024-09-11T08:13:40.434 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:13:40 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:13:41.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:41 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:41.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:41 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:13:41.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-114"}]': finished 2024-09-11T08:13:41.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:41 smithi102 ceph-mon[41371]: osdmap e570: 8 total, 8 up, 8 in 2024-09-11T08:13:41.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:41 smithi102 ceph-mon[41371]: pgmap v885: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-11T08:13:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:41 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:41 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:13:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-114"}]': finished 2024-09-11T08:13:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:41 smithi102 ceph-mon[29849]: osdmap e570: 8 total, 8 up, 8 in 2024-09-11T08:13:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:41 smithi102 ceph-mon[29849]: pgmap v885: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-11T08:13:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:41 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:41 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:13:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-114"}]': finished 2024-09-11T08:13:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:41 smithi146 ceph-mon[33158]: osdmap e570: 8 total, 8 up, 8 in 2024-09-11T08:13:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:41 smithi146 ceph-mon[33158]: pgmap v885: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-11T08:13:42.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:42 smithi102 ceph-mon[41371]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:13:42.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:42 smithi102 ceph-mon[41371]: osdmap e571: 8 total, 8 up, 8 in 2024-09-11T08:13:42.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:42 smithi102 ceph-mon[29849]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:13:42.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:42 smithi102 ceph-mon[29849]: osdmap e571: 8 total, 8 up, 8 in 2024-09-11T08:13:42.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:42 smithi146 ceph-mon[33158]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:13:42.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:42 smithi146 ceph-mon[33158]: osdmap e571: 8 total, 8 up, 8 in 2024-09-11T08:13:43.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:43 smithi102 ceph-mon[41371]: osdmap e572: 8 total, 8 up, 8 in 2024-09-11T08:13:43.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:43 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:13:43.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:43 smithi102 ceph-mon[41371]: pgmap v888: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:43.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:43 smithi102 ceph-mon[29849]: osdmap e572: 8 total, 8 up, 8 in 2024-09-11T08:13:43.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:43 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:13:43.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:43 smithi102 ceph-mon[29849]: pgmap v888: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:43.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:43 smithi146 ceph-mon[33158]: osdmap e572: 8 total, 8 up, 8 in 2024-09-11T08:13:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:43 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:13:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:43 smithi146 ceph-mon[33158]: pgmap v888: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:44.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:44 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:13:44.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:44 smithi102 ceph-mon[41371]: osdmap e573: 8 total, 8 up, 8 in 2024-09-11T08:13:44.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:44 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:13:44.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:44 smithi102 ceph-mon[29849]: osdmap e573: 8 total, 8 up, 8 in 2024-09-11T08:13:44.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:44 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:13:44.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:44 smithi146 ceph-mon[33158]: osdmap e573: 8 total, 8 up, 8 in 2024-09-11T08:13:44.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:13:44 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:13:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:13:45.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:45 smithi102 ceph-mon[41371]: osdmap e574: 8 total, 8 up, 8 in 2024-09-11T08:13:45.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:45 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:13:45.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:45 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:45.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:45 smithi102 ceph-mon[41371]: pgmap v891: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:13:45.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:45 smithi102 ceph-mon[29849]: osdmap e574: 8 total, 8 up, 8 in 2024-09-11T08:13:45.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:45 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:13:45.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:45 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:45.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:45 smithi102 ceph-mon[29849]: pgmap v891: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:13:45.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:45 smithi146 ceph-mon[33158]: osdmap e574: 8 total, 8 up, 8 in 2024-09-11T08:13:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:45 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:13:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:45 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:45 smithi146 ceph-mon[33158]: pgmap v891: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:13:46.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:46 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:13:46.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:46 smithi102 ceph-mon[41371]: osdmap e575: 8 total, 8 up, 8 in 2024-09-11T08:13:46.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:46 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-116"}]: dispatch 2024-09-11T08:13:46.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:13:46.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:46 smithi102 ceph-mon[29849]: osdmap e575: 8 total, 8 up, 8 in 2024-09-11T08:13:46.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-116"}]: dispatch 2024-09-11T08:13:46.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:46 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:13:46.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:46 smithi146 ceph-mon[33158]: osdmap e575: 8 total, 8 up, 8 in 2024-09-11T08:13:46.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:46 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-116"}]: dispatch 2024-09-11T08:13:47.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-116"}]': finished 2024-09-11T08:13:47.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:47 smithi102 ceph-mon[41371]: osdmap e576: 8 total, 8 up, 8 in 2024-09-11T08:13:47.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-116", "mode": "writeback"}]: dispatch 2024-09-11T08:13:47.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:47 smithi102 ceph-mon[41371]: pgmap v894: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:13:47.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:47 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:13:47.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:47 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:13:47.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-116", "mode": "writeback"}]': finished 2024-09-11T08:13:47.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:47 smithi102 ceph-mon[41371]: osdmap e577: 8 total, 8 up, 8 in 2024-09-11T08:13:47.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-116"}]': finished 2024-09-11T08:13:47.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:47 smithi102 ceph-mon[29849]: osdmap e576: 8 total, 8 up, 8 in 2024-09-11T08:13:47.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-116", "mode": "writeback"}]: dispatch 2024-09-11T08:13:47.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:47 smithi102 ceph-mon[29849]: pgmap v894: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:13:47.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:47 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:13:47.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:47 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:13:47.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-116", "mode": "writeback"}]': finished 2024-09-11T08:13:47.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:47 smithi102 ceph-mon[29849]: osdmap e577: 8 total, 8 up, 8 in 2024-09-11T08:13:47.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-116"}]': finished 2024-09-11T08:13:47.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:47 smithi146 ceph-mon[33158]: osdmap e576: 8 total, 8 up, 8 in 2024-09-11T08:13:47.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-116", "mode": "writeback"}]: dispatch 2024-09-11T08:13:47.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:47 smithi146 ceph-mon[33158]: pgmap v894: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:13:47.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:47 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:13:47.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:47 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:13:47.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-116", "mode": "writeback"}]': finished 2024-09-11T08:13:47.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:47 smithi146 ceph-mon[33158]: osdmap e577: 8 total, 8 up, 8 in 2024-09-11T08:13:48.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "244.7"}]: dispatch 2024-09-11T08:13:48.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:48 smithi102 ceph-mon[41371]: from='mon.0 v1:172.21.15.102:0/3661634874' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "244.7"}]: dispatch 2024-09-11T08:13:48.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:48 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:13:48.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:48 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:13:48.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:48 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:13:48.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:48 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:13:48.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "244.7"}]: dispatch 2024-09-11T08:13:48.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:48 smithi102 ceph-mon[29849]: from='mon.0 v1:172.21.15.102:0/3661634874' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "244.7"}]: dispatch 2024-09-11T08:13:48.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:48 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:13:48.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:48 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:13:48.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:48 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:13:48.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:48 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:13:48.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "244.7"}]: dispatch 2024-09-11T08:13:48.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:48 smithi146 ceph-mon[33158]: from='mon.0 v1:172.21.15.102:0/3661634874' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "244.7"}]: dispatch 2024-09-11T08:13:48.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:48 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:13:48.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:48 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:13:48.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:48 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:13:48.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:48 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:13:49.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:49 smithi102 ceph-mon[41371]: pgmap v896: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-09-11T08:13:49.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:49 smithi102 ceph-mon[29849]: pgmap v896: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-09-11T08:13:49.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:49 smithi146 ceph-mon[33158]: pgmap v896: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-09-11T08:13:50.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:13:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:13:49] "GET /metrics HTTP/1.1" 200 228693 "" "Prometheus/2.43.0" 2024-09-11T08:13:50.434 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:13:50 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:13:51.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:51 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:51.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:51 smithi102 ceph-mon[29849]: pgmap v897: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-11T08:13:51.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:51 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:51.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:51 smithi102 ceph-mon[41371]: pgmap v897: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-11T08:13:51.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:51 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:13:51.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:51 smithi146 ceph-mon[33158]: pgmap v897: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-11T08:13:53.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:52 smithi102 ceph-mon[41371]: 244.7 scrub starts 2024-09-11T08:13:53.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:52 smithi102 ceph-mon[41371]: 244.7 scrub ok 2024-09-11T08:13:53.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:52 smithi102 ceph-mon[29849]: 244.7 scrub starts 2024-09-11T08:13:53.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:52 smithi102 ceph-mon[29849]: 244.7 scrub ok 2024-09-11T08:13:53.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:52 smithi146 ceph-mon[33158]: 244.7 scrub starts 2024-09-11T08:13:53.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:52 smithi146 ceph-mon[33158]: 244.7 scrub ok 2024-09-11T08:13:54.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:53 smithi102 ceph-mon[41371]: pgmap v898: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 958 B/s wr, 2 op/s 2024-09-11T08:13:54.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:53 smithi102 ceph-mon[29849]: pgmap v898: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 958 B/s wr, 2 op/s 2024-09-11T08:13:54.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:53 smithi146 ceph-mon[33158]: pgmap v898: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 958 B/s wr, 2 op/s 2024-09-11T08:13:54.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:13:54 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:13:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:13:56.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:55 smithi102 ceph-mon[41371]: pgmap v899: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 845 B/s rd, 1 op/s 2024-09-11T08:13:56.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:55 smithi102 ceph-mon[29849]: pgmap v899: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 845 B/s rd, 1 op/s 2024-09-11T08:13:56.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:55 smithi146 ceph-mon[33158]: pgmap v899: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 845 B/s rd, 1 op/s 2024-09-11T08:13:57.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:57 smithi102 ceph-mon[41371]: pgmap v900: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:57.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:57 smithi102 ceph-mon[29849]: pgmap v900: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:57.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:57 smithi146 ceph-mon[33158]: pgmap v900: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:13:59.895 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:13:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:13:59] "GET /metrics HTTP/1.1" 200 228685 "" "Prometheus/2.43.0" 2024-09-11T08:13:59.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:59 smithi102 ceph-mon[29849]: pgmap v901: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:13:59.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:59 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:13:59.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:13:59 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:59.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:59 smithi102 ceph-mon[41371]: pgmap v901: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:13:59.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:59 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:13:59.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:13:59 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:13:59.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:59 smithi146 ceph-mon[33158]: pgmap v901: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-11T08:13:59.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:59 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:13:59.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:13:59 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:00.433 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:14:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:14:01.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:01 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:01.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:01 smithi102 ceph-mon[29849]: pgmap v902: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T08:14:01.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:01 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:01.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:01 smithi102 ceph-mon[41371]: pgmap v902: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T08:14:01.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:01 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:01.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:01 smithi146 ceph-mon[33158]: pgmap v902: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-11T08:14:03.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:03 smithi102 ceph-mon[29849]: pgmap v903: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:03.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:03 smithi102 ceph-mon[29849]: osdmap e578: 8 total, 8 up, 8 in 2024-09-11T08:14:03.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:03 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:14:03.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:03 smithi102 ceph-mon[41371]: pgmap v903: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:03.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:03 smithi102 ceph-mon[41371]: osdmap e578: 8 total, 8 up, 8 in 2024-09-11T08:14:03.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:03 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:14:03.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:03 smithi146 ceph-mon[33158]: pgmap v903: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:14:03.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:03 smithi146 ceph-mon[33158]: osdmap e578: 8 total, 8 up, 8 in 2024-09-11T08:14:03.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:03 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:14:04.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:04 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:14:04.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:04 smithi102 ceph-mon[29849]: osdmap e579: 8 total, 8 up, 8 in 2024-09-11T08:14:04.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:04 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-116"}]: dispatch 2024-09-11T08:14:04.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:04 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:14:04.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:04 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:14:04.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:04 smithi102 ceph-mon[41371]: osdmap e579: 8 total, 8 up, 8 in 2024-09-11T08:14:04.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:04 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-116"}]: dispatch 2024-09-11T08:14:04.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:04 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:14:04.937 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:14:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:14:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:14:04.937 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:04 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:14:04.937 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:04 smithi146 ceph-mon[33158]: osdmap e579: 8 total, 8 up, 8 in 2024-09-11T08:14:04.937 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:04 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-116"}]: dispatch 2024-09-11T08:14:04.938 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:04 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:14:05.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:05 smithi102 ceph-mon[41371]: pgmap v906: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:14:05.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:05 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-116"}]': finished 2024-09-11T08:14:05.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:05 smithi102 ceph-mon[41371]: osdmap e580: 8 total, 8 up, 8 in 2024-09-11T08:14:05.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:05 smithi102 ceph-mon[29849]: pgmap v906: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:14:05.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:05 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-116"}]': finished 2024-09-11T08:14:05.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:05 smithi102 ceph-mon[29849]: osdmap e580: 8 total, 8 up, 8 in 2024-09-11T08:14:05.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:05 smithi146 ceph-mon[33158]: pgmap v906: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:14:05.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:05 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-116"}]': finished 2024-09-11T08:14:05.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:05 smithi146 ceph-mon[33158]: osdmap e580: 8 total, 8 up, 8 in 2024-09-11T08:14:06.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:06 smithi146 ceph-mon[33158]: osdmap e581: 8 total, 8 up, 8 in 2024-09-11T08:14:07.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:06 smithi102 ceph-mon[41371]: osdmap e581: 8 total, 8 up, 8 in 2024-09-11T08:14:07.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:06 smithi102 ceph-mon[29849]: osdmap e581: 8 total, 8 up, 8 in 2024-09-11T08:14:07.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:07 smithi146 ceph-mon[33158]: pgmap v909: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-11T08:14:07.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:07 smithi146 ceph-mon[33158]: osdmap e582: 8 total, 8 up, 8 in 2024-09-11T08:14:07.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:14:07.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:14:07.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:07 smithi146 ceph-mon[33158]: osdmap e583: 8 total, 8 up, 8 in 2024-09-11T08:14:08.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:07 smithi102 ceph-mon[41371]: pgmap v909: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-11T08:14:08.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:07 smithi102 ceph-mon[41371]: osdmap e582: 8 total, 8 up, 8 in 2024-09-11T08:14:08.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:14:08.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:14:08.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:07 smithi102 ceph-mon[41371]: osdmap e583: 8 total, 8 up, 8 in 2024-09-11T08:14:08.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:07 smithi102 ceph-mon[29849]: pgmap v909: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-11T08:14:08.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:07 smithi102 ceph-mon[29849]: osdmap e582: 8 total, 8 up, 8 in 2024-09-11T08:14:08.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:14:08.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:14:08.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:07 smithi102 ceph-mon[29849]: osdmap e583: 8 total, 8 up, 8 in 2024-09-11T08:14:09.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:08 smithi102 ceph-mon[41371]: osdmap e584: 8 total, 8 up, 8 in 2024-09-11T08:14:09.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:08 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:14:09.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:08 smithi102 ceph-mon[29849]: osdmap e584: 8 total, 8 up, 8 in 2024-09-11T08:14:09.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:08 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:14:09.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:08 smithi146 ceph-mon[33158]: osdmap e584: 8 total, 8 up, 8 in 2024-09-11T08:14:09.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:08 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:14:10.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:14:09 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:14:09] "GET /metrics HTTP/1.1" 200 228685 "" "Prometheus/2.43.0" 2024-09-11T08:14:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:10 smithi102 ceph-mon[41371]: pgmap v913: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 780 B/s rd, 1 op/s 2024-09-11T08:14:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:14:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:10 smithi102 ceph-mon[41371]: osdmap e585: 8 total, 8 up, 8 in 2024-09-11T08:14:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-118"}]: dispatch 2024-09-11T08:14:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:10 smithi102 ceph-mon[29849]: pgmap v913: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 780 B/s rd, 1 op/s 2024-09-11T08:14:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:14:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:10 smithi102 ceph-mon[29849]: osdmap e585: 8 total, 8 up, 8 in 2024-09-11T08:14:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-118"}]: dispatch 2024-09-11T08:14:10.434 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:14:10 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:14:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:10 smithi146 ceph-mon[33158]: pgmap v913: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 780 B/s rd, 1 op/s 2024-09-11T08:14:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:14:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:10 smithi146 ceph-mon[33158]: osdmap e585: 8 total, 8 up, 8 in 2024-09-11T08:14:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-118"}]: dispatch 2024-09-11T08:14:11.013 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:11 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-118"}]': finished 2024-09-11T08:14:11.013 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:11 smithi102 ceph-mon[29849]: osdmap e586: 8 total, 8 up, 8 in 2024-09-11T08:14:11.013 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:11 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-118", "mode": "writeback"}]: dispatch 2024-09-11T08:14:11.013 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:11 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:11.013 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:11 smithi102 ceph-mon[29849]: pgmap v916: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 522 B/s wr, 1 op/s 2024-09-11T08:14:11.013 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:11 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:14:11.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:11 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-118", "mode": "writeback"}]': finished 2024-09-11T08:14:11.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:11 smithi102 ceph-mon[29849]: osdmap e587: 8 total, 8 up, 8 in 2024-09-11T08:14:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:11 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-118"}]': finished 2024-09-11T08:14:11.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:11 smithi102 ceph-mon[41371]: osdmap e586: 8 total, 8 up, 8 in 2024-09-11T08:14:11.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:11 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-118", "mode": "writeback"}]: dispatch 2024-09-11T08:14:11.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:11 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:11.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:11 smithi102 ceph-mon[41371]: pgmap v916: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 522 B/s wr, 1 op/s 2024-09-11T08:14:11.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:11 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:14:11.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:11 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-118", "mode": "writeback"}]': finished 2024-09-11T08:14:11.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:11 smithi102 ceph-mon[41371]: osdmap e587: 8 total, 8 up, 8 in 2024-09-11T08:14:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:11 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-118"}]': finished 2024-09-11T08:14:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:11 smithi146 ceph-mon[33158]: osdmap e586: 8 total, 8 up, 8 in 2024-09-11T08:14:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:11 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-118", "mode": "writeback"}]: dispatch 2024-09-11T08:14:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:11 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:11.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:11 smithi146 ceph-mon[33158]: pgmap v916: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 522 B/s wr, 1 op/s 2024-09-11T08:14:11.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:11 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:14:11.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:11 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-118", "mode": "writeback"}]': finished 2024-09-11T08:14:11.435 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:11 smithi146 ceph-mon[33158]: osdmap e587: 8 total, 8 up, 8 in 2024-09-11T08:14:13.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:13 smithi102 ceph-mon[41371]: osdmap e588: 8 total, 8 up, 8 in 2024-09-11T08:14:13.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:14:13.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:13 smithi102 ceph-mon[41371]: pgmap v919: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:14:13.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:13 smithi102 ceph-mon[29849]: osdmap e588: 8 total, 8 up, 8 in 2024-09-11T08:14:13.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:14:13.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:13 smithi102 ceph-mon[29849]: pgmap v919: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:14:13.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:13 smithi146 ceph-mon[33158]: osdmap e588: 8 total, 8 up, 8 in 2024-09-11T08:14:13.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:14:13.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:13 smithi146 ceph-mon[33158]: pgmap v919: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:14:14.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:14:14.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:14 smithi102 ceph-mon[29849]: osdmap e589: 8 total, 8 up, 8 in 2024-09-11T08:14:14.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-118"}]: dispatch 2024-09-11T08:14:14.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:14:14.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:14 smithi102 ceph-mon[41371]: osdmap e589: 8 total, 8 up, 8 in 2024-09-11T08:14:14.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-118"}]: dispatch 2024-09-11T08:14:14.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:14:14.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:14 smithi146 ceph-mon[33158]: osdmap e589: 8 total, 8 up, 8 in 2024-09-11T08:14:14.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-118"}]: dispatch 2024-09-11T08:14:14.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:14:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:14:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:14:15.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:15 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:14:15.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:15 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-118"}]': finished 2024-09-11T08:14:15.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:15 smithi102 ceph-mon[41371]: osdmap e590: 8 total, 8 up, 8 in 2024-09-11T08:14:15.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:15 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:14:15.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:15.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:15 smithi102 ceph-mon[41371]: pgmap v922: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:15.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:15 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:14:15.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:15 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-118"}]': finished 2024-09-11T08:14:15.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:15 smithi102 ceph-mon[29849]: osdmap e590: 8 total, 8 up, 8 in 2024-09-11T08:14:15.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:15 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:14:15.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:15.647 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:15 smithi102 ceph-mon[29849]: pgmap v922: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:15.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:15 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:14:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:15 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-118"}]': finished 2024-09-11T08:14:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:15 smithi146 ceph-mon[33158]: osdmap e590: 8 total, 8 up, 8 in 2024-09-11T08:14:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:15 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:14:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:15 smithi146 ceph-mon[33158]: pgmap v922: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:16.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:16 smithi102 ceph-mon[41371]: osdmap e591: 8 total, 8 up, 8 in 2024-09-11T08:14:16.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:16 smithi102 ceph-mon[41371]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:16.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:16 smithi102 ceph-mon[41371]: osdmap e592: 8 total, 8 up, 8 in 2024-09-11T08:14:16.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:14:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:16 smithi102 ceph-mon[29849]: osdmap e591: 8 total, 8 up, 8 in 2024-09-11T08:14:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:16 smithi102 ceph-mon[29849]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:16 smithi102 ceph-mon[29849]: osdmap e592: 8 total, 8 up, 8 in 2024-09-11T08:14:16.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:14:16.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:16 smithi146 ceph-mon[33158]: osdmap e591: 8 total, 8 up, 8 in 2024-09-11T08:14:16.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:16 smithi146 ceph-mon[33158]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:16.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:16 smithi146 ceph-mon[33158]: osdmap e592: 8 total, 8 up, 8 in 2024-09-11T08:14:16.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:14:17.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:17 smithi102 ceph-mon[41371]: pgmap v925: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:17.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:14:17.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:17 smithi102 ceph-mon[41371]: osdmap e593: 8 total, 8 up, 8 in 2024-09-11T08:14:17.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:14:17.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:17 smithi102 ceph-mon[29849]: pgmap v925: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:17.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:14:17.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:17 smithi102 ceph-mon[29849]: osdmap e593: 8 total, 8 up, 8 in 2024-09-11T08:14:17.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:14:17.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:17 smithi146 ceph-mon[33158]: pgmap v925: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:17.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:14:17.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:17 smithi146 ceph-mon[33158]: osdmap e593: 8 total, 8 up, 8 in 2024-09-11T08:14:17.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:14:19.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:14:19.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:19 smithi102 ceph-mon[29849]: osdmap e594: 8 total, 8 up, 8 in 2024-09-11T08:14:19.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-120"}]: dispatch 2024-09-11T08:14:19.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:19 smithi102 ceph-mon[29849]: pgmap v928: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:19.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:14:19.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:19 smithi102 ceph-mon[41371]: osdmap e594: 8 total, 8 up, 8 in 2024-09-11T08:14:19.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-120"}]: dispatch 2024-09-11T08:14:19.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:19 smithi102 ceph-mon[41371]: pgmap v928: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:19.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:14:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:19 smithi146 ceph-mon[33158]: osdmap e594: 8 total, 8 up, 8 in 2024-09-11T08:14:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-120"}]: dispatch 2024-09-11T08:14:19.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:19 smithi146 ceph-mon[33158]: pgmap v928: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:20.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-120"}]': finished 2024-09-11T08:14:20.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:20 smithi102 ceph-mon[29849]: osdmap e595: 8 total, 8 up, 8 in 2024-09-11T08:14:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-120", "mode": "writeback"}]: dispatch 2024-09-11T08:14:20.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-120"}]': finished 2024-09-11T08:14:20.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:20 smithi102 ceph-mon[41371]: osdmap e595: 8 total, 8 up, 8 in 2024-09-11T08:14:20.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-120", "mode": "writeback"}]: dispatch 2024-09-11T08:14:20.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:14:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:14:19] "GET /metrics HTTP/1.1" 200 228676 "" "Prometheus/2.43.0" 2024-09-11T08:14:20.434 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:14:20 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:14:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-120"}]': finished 2024-09-11T08:14:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:20 smithi146 ceph-mon[33158]: osdmap e595: 8 total, 8 up, 8 in 2024-09-11T08:14:20.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-120", "mode": "writeback"}]: dispatch 2024-09-11T08:14:21.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:21 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:14:21.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:21 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-120", "mode": "writeback"}]': finished 2024-09-11T08:14:21.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:21 smithi102 ceph-mon[41371]: osdmap e596: 8 total, 8 up, 8 in 2024-09-11T08:14:21.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:21 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:21.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:21 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:14:21.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:21 smithi102 ceph-mon[41371]: pgmap v931: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-11T08:14:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:21 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:14:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:21 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-120", "mode": "writeback"}]': finished 2024-09-11T08:14:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:21 smithi102 ceph-mon[29849]: osdmap e596: 8 total, 8 up, 8 in 2024-09-11T08:14:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:21 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:21 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:14:21.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:21 smithi102 ceph-mon[29849]: pgmap v931: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-11T08:14:21.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:21 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:14:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:21 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-120", "mode": "writeback"}]': finished 2024-09-11T08:14:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:21 smithi146 ceph-mon[33158]: osdmap e596: 8 total, 8 up, 8 in 2024-09-11T08:14:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:21 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:21 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:14:21.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:21 smithi146 ceph-mon[33158]: pgmap v931: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-11T08:14:22.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:14:22.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:22 smithi102 ceph-mon[29849]: osdmap e597: 8 total, 8 up, 8 in 2024-09-11T08:14:22.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-120"}]: dispatch 2024-09-11T08:14:22.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:22 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:22.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:22 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:14:22.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-120"}]': finished 2024-09-11T08:14:22.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:22 smithi102 ceph-mon[29849]: osdmap e598: 8 total, 8 up, 8 in 2024-09-11T08:14:22.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:14:22.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:22 smithi102 ceph-mon[41371]: osdmap e597: 8 total, 8 up, 8 in 2024-09-11T08:14:22.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-120"}]: dispatch 2024-09-11T08:14:22.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:22 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:22.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:22 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:14:22.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-120"}]': finished 2024-09-11T08:14:22.398 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:22 smithi102 ceph-mon[41371]: osdmap e598: 8 total, 8 up, 8 in 2024-09-11T08:14:22.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:14:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:22 smithi146 ceph-mon[33158]: osdmap e597: 8 total, 8 up, 8 in 2024-09-11T08:14:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-120"}]: dispatch 2024-09-11T08:14:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:22 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:22 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:14:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-120"}]': finished 2024-09-11T08:14:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:22 smithi146 ceph-mon[33158]: osdmap e598: 8 total, 8 up, 8 in 2024-09-11T08:14:24.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:24 smithi102 ceph-mon[29849]: pgmap v934: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-11T08:14:24.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:24 smithi102 ceph-mon[29849]: osdmap e599: 8 total, 8 up, 8 in 2024-09-11T08:14:24.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:24 smithi102 ceph-mon[41371]: pgmap v934: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-11T08:14:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:24 smithi102 ceph-mon[41371]: osdmap e599: 8 total, 8 up, 8 in 2024-09-11T08:14:24.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:24 smithi146 ceph-mon[33158]: pgmap v934: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-11T08:14:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:24 smithi146 ceph-mon[33158]: osdmap e599: 8 total, 8 up, 8 in 2024-09-11T08:14:24.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:14:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:14:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:14:25.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:25 smithi102 ceph-mon[29849]: osdmap e600: 8 total, 8 up, 8 in 2024-09-11T08:14:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:14:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:25 smithi102 ceph-mon[29849]: pgmap v937: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:14:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:25 smithi102 ceph-mon[29849]: osdmap e601: 8 total, 8 up, 8 in 2024-09-11T08:14:25.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:25 smithi102 ceph-mon[41371]: osdmap e600: 8 total, 8 up, 8 in 2024-09-11T08:14:25.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:14:25.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:25 smithi102 ceph-mon[41371]: pgmap v937: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:25.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:14:25.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:25 smithi102 ceph-mon[41371]: osdmap e601: 8 total, 8 up, 8 in 2024-09-11T08:14:25.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:25 smithi146 ceph-mon[33158]: osdmap e600: 8 total, 8 up, 8 in 2024-09-11T08:14:25.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:14:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:25 smithi146 ceph-mon[33158]: pgmap v937: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:14:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:25 smithi146 ceph-mon[33158]: osdmap e601: 8 total, 8 up, 8 in 2024-09-11T08:14:26.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:14:26.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:14:26.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:26 smithi102 ceph-mon[29849]: osdmap e602: 8 total, 8 up, 8 in 2024-09-11T08:14:26.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-122"}]: dispatch 2024-09-11T08:14:26.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:14:26.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:14:26.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:26 smithi102 ceph-mon[41371]: osdmap e602: 8 total, 8 up, 8 in 2024-09-11T08:14:26.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-122"}]: dispatch 2024-09-11T08:14:26.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:14:26.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:14:26.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:26 smithi146 ceph-mon[33158]: osdmap e602: 8 total, 8 up, 8 in 2024-09-11T08:14:26.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-122"}]: dispatch 2024-09-11T08:14:27.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:27 smithi102 ceph-mon[29849]: pgmap v940: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:14:27.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:27 smithi102 ceph-mon[29849]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:27.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-122"}]': finished 2024-09-11T08:14:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:27 smithi102 ceph-mon[29849]: osdmap e603: 8 total, 8 up, 8 in 2024-09-11T08:14:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-122", "mode": "writeback"}]: dispatch 2024-09-11T08:14:27.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:27 smithi102 ceph-mon[41371]: pgmap v940: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:14:27.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:27 smithi102 ceph-mon[41371]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:27.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-122"}]': finished 2024-09-11T08:14:27.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:27 smithi102 ceph-mon[41371]: osdmap e603: 8 total, 8 up, 8 in 2024-09-11T08:14:27.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-122", "mode": "writeback"}]: dispatch 2024-09-11T08:14:27.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:27 smithi146 ceph-mon[33158]: pgmap v940: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:14:27.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:27 smithi146 ceph-mon[33158]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:27.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-122"}]': finished 2024-09-11T08:14:27.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:27 smithi146 ceph-mon[33158]: osdmap e603: 8 total, 8 up, 8 in 2024-09-11T08:14:27.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-122", "mode": "writeback"}]: dispatch 2024-09-11T08:14:28.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:28 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:14:28.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-122", "mode": "writeback"}]': finished 2024-09-11T08:14:28.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:28 smithi102 ceph-mon[41371]: osdmap e604: 8 total, 8 up, 8 in 2024-09-11T08:14:28.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:28 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:14:28.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-122", "mode": "writeback"}]': finished 2024-09-11T08:14:28.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:28 smithi102 ceph-mon[29849]: osdmap e604: 8 total, 8 up, 8 in 2024-09-11T08:14:28.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:28 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:14:28.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-122", "mode": "writeback"}]': finished 2024-09-11T08:14:28.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:28 smithi146 ceph-mon[33158]: osdmap e604: 8 total, 8 up, 8 in 2024-09-11T08:14:29.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:14:29.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:29 smithi102 ceph-mon[41371]: pgmap v943: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:14:29.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:14:29.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:29 smithi102 ceph-mon[29849]: pgmap v943: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:14:29.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:14:29.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:29 smithi146 ceph-mon[33158]: pgmap v943: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:14:30.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:14:29 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:14:29] "GET /metrics HTTP/1.1" 200 228679 "" "Prometheus/2.43.0" 2024-09-11T08:14:30.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:30 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:14:30.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:30 smithi102 ceph-mon[41371]: osdmap e605: 8 total, 8 up, 8 in 2024-09-11T08:14:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:30 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-122"}]: dispatch 2024-09-11T08:14:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:30 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:14:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:30 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:30.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:30 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:14:30.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:30 smithi102 ceph-mon[29849]: osdmap e605: 8 total, 8 up, 8 in 2024-09-11T08:14:30.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:30 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-122"}]: dispatch 2024-09-11T08:14:30.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:30 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:14:30.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:30 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:30.433 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:14:30 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:14:30.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:30 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:14:30.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:30 smithi146 ceph-mon[33158]: osdmap e605: 8 total, 8 up, 8 in 2024-09-11T08:14:30.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:30 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-122"}]: dispatch 2024-09-11T08:14:30.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:30 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:14:30.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:30 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:31.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:31 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:14:31.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:31 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-122"}]': finished 2024-09-11T08:14:31.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:31 smithi102 ceph-mon[29849]: osdmap e606: 8 total, 8 up, 8 in 2024-09-11T08:14:31.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:31 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:31.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:31 smithi102 ceph-mon[29849]: pgmap v946: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-09-11T08:14:31.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:31 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:14:31.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:31 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-122"}]': finished 2024-09-11T08:14:31.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:31 smithi102 ceph-mon[41371]: osdmap e606: 8 total, 8 up, 8 in 2024-09-11T08:14:31.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:31 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:31.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:31 smithi102 ceph-mon[41371]: pgmap v946: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-09-11T08:14:31.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:31 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:14:31.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:31 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-122"}]': finished 2024-09-11T08:14:31.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:31 smithi146 ceph-mon[33158]: osdmap e606: 8 total, 8 up, 8 in 2024-09-11T08:14:31.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:31 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:31.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:31 smithi146 ceph-mon[33158]: pgmap v946: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 3 op/s 2024-09-11T08:14:32.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:32 smithi102 ceph-mon[29849]: osdmap e607: 8 total, 8 up, 8 in 2024-09-11T08:14:32.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:32 smithi102 ceph-mon[41371]: osdmap e607: 8 total, 8 up, 8 in 2024-09-11T08:14:32.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:32 smithi146 ceph-mon[33158]: osdmap e607: 8 total, 8 up, 8 in 2024-09-11T08:14:33.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:33 smithi102 ceph-mon[41371]: osdmap e608: 8 total, 8 up, 8 in 2024-09-11T08:14:33.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:14:33.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:33 smithi102 ceph-mon[41371]: pgmap v949: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:14:33.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:33 smithi102 ceph-mon[29849]: osdmap e608: 8 total, 8 up, 8 in 2024-09-11T08:14:33.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:14:33.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:33 smithi102 ceph-mon[29849]: pgmap v949: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:14:33.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:33 smithi146 ceph-mon[33158]: osdmap e608: 8 total, 8 up, 8 in 2024-09-11T08:14:33.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:14:33.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:33 smithi146 ceph-mon[33158]: pgmap v949: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:14:34.336 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:34 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:14:34.336 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:34 smithi146 ceph-mon[33158]: osdmap e609: 8 total, 8 up, 8 in 2024-09-11T08:14:34.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:34 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:14:34.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:34 smithi102 ceph-mon[41371]: osdmap e609: 8 total, 8 up, 8 in 2024-09-11T08:14:34.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:34 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:14:34.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:34 smithi102 ceph-mon[29849]: osdmap e609: 8 total, 8 up, 8 in 2024-09-11T08:14:34.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:14:34 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:14:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:14:35.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:35 smithi102 ceph-mon[41371]: osdmap e610: 8 total, 8 up, 8 in 2024-09-11T08:14:35.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:14:35.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:35 smithi102 ceph-mon[41371]: pgmap v952: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:35 smithi102 ceph-mon[29849]: osdmap e610: 8 total, 8 up, 8 in 2024-09-11T08:14:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:14:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:35 smithi102 ceph-mon[29849]: pgmap v952: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:35.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:35 smithi146 ceph-mon[33158]: osdmap e610: 8 total, 8 up, 8 in 2024-09-11T08:14:35.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:14:35.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:35 smithi146 ceph-mon[33158]: pgmap v952: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:36.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:36 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:36.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:36 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:14:36.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:36 smithi102 ceph-mon[41371]: osdmap e611: 8 total, 8 up, 8 in 2024-09-11T08:14:36.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:36 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-124"}]: dispatch 2024-09-11T08:14:36.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:36 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:36.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:36 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:14:36.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:36 smithi102 ceph-mon[29849]: osdmap e611: 8 total, 8 up, 8 in 2024-09-11T08:14:36.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:36 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-124"}]: dispatch 2024-09-11T08:14:36.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:36 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:36.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:36 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:14:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:36 smithi146 ceph-mon[33158]: osdmap e611: 8 total, 8 up, 8 in 2024-09-11T08:14:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:36 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-124"}]: dispatch 2024-09-11T08:14:37.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-124"}]': finished 2024-09-11T08:14:37.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:37 smithi102 ceph-mon[41371]: osdmap e612: 8 total, 8 up, 8 in 2024-09-11T08:14:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-124", "mode": "writeback"}]: dispatch 2024-09-11T08:14:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:37 smithi102 ceph-mon[41371]: pgmap v955: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:14:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:37 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:14:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-124", "mode": "writeback"}]': finished 2024-09-11T08:14:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:37 smithi102 ceph-mon[41371]: osdmap e613: 8 total, 8 up, 8 in 2024-09-11T08:14:37.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-124"}]': finished 2024-09-11T08:14:37.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:37 smithi102 ceph-mon[29849]: osdmap e612: 8 total, 8 up, 8 in 2024-09-11T08:14:37.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-124", "mode": "writeback"}]: dispatch 2024-09-11T08:14:37.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:37 smithi102 ceph-mon[29849]: pgmap v955: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:14:37.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:37 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:14:37.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-124", "mode": "writeback"}]': finished 2024-09-11T08:14:37.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:37 smithi102 ceph-mon[29849]: osdmap e613: 8 total, 8 up, 8 in 2024-09-11T08:14:37.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-124"}]': finished 2024-09-11T08:14:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:37 smithi146 ceph-mon[33158]: osdmap e612: 8 total, 8 up, 8 in 2024-09-11T08:14:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-124", "mode": "writeback"}]: dispatch 2024-09-11T08:14:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:37 smithi146 ceph-mon[33158]: pgmap v955: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:14:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:37 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:14:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-124", "mode": "writeback"}]': finished 2024-09-11T08:14:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:37 smithi146 ceph-mon[33158]: osdmap e613: 8 total, 8 up, 8 in 2024-09-11T08:14:39.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:39 smithi102 ceph-mon[29849]: osdmap e614: 8 total, 8 up, 8 in 2024-09-11T08:14:39.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:39 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:14:39.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:39 smithi102 ceph-mon[29849]: pgmap v958: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:14:39.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:39 smithi102 ceph-mon[41371]: osdmap e614: 8 total, 8 up, 8 in 2024-09-11T08:14:39.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:39 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:14:39.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:39 smithi102 ceph-mon[41371]: pgmap v958: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:14:39.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:39 smithi146 ceph-mon[33158]: osdmap e614: 8 total, 8 up, 8 in 2024-09-11T08:14:39.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:39 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:14:39.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:39 smithi146 ceph-mon[33158]: pgmap v958: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-11T08:14:40.109 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:14:39 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:14:39] "GET /metrics HTTP/1.1" 200 228679 "" "Prometheus/2.43.0" 2024-09-11T08:14:40.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:14:40.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:40 smithi102 ceph-mon[41371]: osdmap e615: 8 total, 8 up, 8 in 2024-09-11T08:14:40.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-124"}]: dispatch 2024-09-11T08:14:40.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:14:40.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:40 smithi102 ceph-mon[29849]: osdmap e615: 8 total, 8 up, 8 in 2024-09-11T08:14:40.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-124"}]: dispatch 2024-09-11T08:14:40.433 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:14:40 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:14:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:14:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:40 smithi146 ceph-mon[33158]: osdmap e615: 8 total, 8 up, 8 in 2024-09-11T08:14:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-124"}]: dispatch 2024-09-11T08:14:41.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:41 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:41.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:41 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:14:41.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-124"}]': finished 2024-09-11T08:14:41.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:41 smithi102 ceph-mon[41371]: osdmap e616: 8 total, 8 up, 8 in 2024-09-11T08:14:41.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:41 smithi102 ceph-mon[41371]: pgmap v961: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-11T08:14:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:41 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:41 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:14:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-124"}]': finished 2024-09-11T08:14:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:41 smithi102 ceph-mon[29849]: osdmap e616: 8 total, 8 up, 8 in 2024-09-11T08:14:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:41 smithi102 ceph-mon[29849]: pgmap v961: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-11T08:14:41.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:41 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:41.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:41 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:14:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-124"}]': finished 2024-09-11T08:14:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:41 smithi146 ceph-mon[33158]: osdmap e616: 8 total, 8 up, 8 in 2024-09-11T08:14:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:41 smithi146 ceph-mon[33158]: pgmap v961: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-11T08:14:42.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:42 smithi102 ceph-mon[41371]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:42.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:42 smithi102 ceph-mon[41371]: osdmap e617: 8 total, 8 up, 8 in 2024-09-11T08:14:42.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:42 smithi102 ceph-mon[41371]: osdmap e618: 8 total, 8 up, 8 in 2024-09-11T08:14:42.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:14:42.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:42 smithi102 ceph-mon[29849]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:42.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:42 smithi102 ceph-mon[29849]: osdmap e617: 8 total, 8 up, 8 in 2024-09-11T08:14:42.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:42 smithi102 ceph-mon[29849]: osdmap e618: 8 total, 8 up, 8 in 2024-09-11T08:14:42.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:14:42.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:42 smithi146 ceph-mon[33158]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:42.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:42 smithi146 ceph-mon[33158]: osdmap e617: 8 total, 8 up, 8 in 2024-09-11T08:14:42.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:42 smithi146 ceph-mon[33158]: osdmap e618: 8 total, 8 up, 8 in 2024-09-11T08:14:42.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:14:44.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:44 smithi102 ceph-mon[41371]: pgmap v964: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:44.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:44 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:14:44.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:44 smithi102 ceph-mon[41371]: osdmap e619: 8 total, 8 up, 8 in 2024-09-11T08:14:44.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:44 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:14:44.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:44 smithi102 ceph-mon[29849]: pgmap v964: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:44.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:44 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:14:44.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:44 smithi102 ceph-mon[29849]: osdmap e619: 8 total, 8 up, 8 in 2024-09-11T08:14:44.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:44 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:14:44.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:44 smithi146 ceph-mon[33158]: pgmap v964: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:44.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:44 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:14:44.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:44 smithi146 ceph-mon[33158]: osdmap e619: 8 total, 8 up, 8 in 2024-09-11T08:14:44.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:44 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:14:44.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:14:44 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:14:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:14:45.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:45 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:14:45.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:45 smithi102 ceph-mon[29849]: osdmap e620: 8 total, 8 up, 8 in 2024-09-11T08:14:45.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:45 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-126"}]: dispatch 2024-09-11T08:14:45.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:45 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:14:45.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:45 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:45.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:45 smithi102 ceph-mon[29849]: pgmap v967: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:45.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:45 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-126"}]': finished 2024-09-11T08:14:45.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:45 smithi102 ceph-mon[29849]: osdmap e621: 8 total, 8 up, 8 in 2024-09-11T08:14:45.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:45 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-126", "mode": "writeback"}]: dispatch 2024-09-11T08:14:45.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:45 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:14:45.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:45 smithi102 ceph-mon[41371]: osdmap e620: 8 total, 8 up, 8 in 2024-09-11T08:14:45.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:45 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-126"}]: dispatch 2024-09-11T08:14:45.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:45 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:14:45.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:45 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:45.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:45 smithi102 ceph-mon[41371]: pgmap v967: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:45.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:45 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-126"}]': finished 2024-09-11T08:14:45.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:45 smithi102 ceph-mon[41371]: osdmap e621: 8 total, 8 up, 8 in 2024-09-11T08:14:45.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:45 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-126", "mode": "writeback"}]: dispatch 2024-09-11T08:14:45.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:45 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:14:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:45 smithi146 ceph-mon[33158]: osdmap e620: 8 total, 8 up, 8 in 2024-09-11T08:14:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:45 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-126"}]: dispatch 2024-09-11T08:14:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:45 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:14:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:45 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:45 smithi146 ceph-mon[33158]: pgmap v967: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:45 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-126"}]': finished 2024-09-11T08:14:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:45 smithi146 ceph-mon[33158]: osdmap e621: 8 total, 8 up, 8 in 2024-09-11T08:14:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:45 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-126", "mode": "writeback"}]: dispatch 2024-09-11T08:14:46.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:46 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:14:46.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:46 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-126", "mode": "writeback"}]': finished 2024-09-11T08:14:46.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:46 smithi146 ceph-mon[33158]: osdmap e622: 8 total, 8 up, 8 in 2024-09-11T08:14:46.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:46 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:14:46.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:46 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:14:46.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-126", "mode": "writeback"}]': finished 2024-09-11T08:14:46.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:46 smithi102 ceph-mon[29849]: osdmap e622: 8 total, 8 up, 8 in 2024-09-11T08:14:46.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:14:46.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:46 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:14:46.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:46 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-126", "mode": "writeback"}]': finished 2024-09-11T08:14:46.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:46 smithi102 ceph-mon[41371]: osdmap e622: 8 total, 8 up, 8 in 2024-09-11T08:14:46.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:46 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:14:47.688 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:47 smithi146 ceph-mon[33158]: pgmap v970: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:47.688 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:47 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:47.688 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:14:47.688 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:47 smithi146 ceph-mon[33158]: osdmap e623: 8 total, 8 up, 8 in 2024-09-11T08:14:47.688 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-126"}]: dispatch 2024-09-11T08:14:47.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:47 smithi102 ceph-mon[41371]: pgmap v970: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:47.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:47 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:47.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:14:47.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:47 smithi102 ceph-mon[41371]: osdmap e623: 8 total, 8 up, 8 in 2024-09-11T08:14:47.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-126"}]: dispatch 2024-09-11T08:14:47.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:47 smithi102 ceph-mon[29849]: pgmap v970: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:47.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:47 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:47.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:14:47.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:47 smithi102 ceph-mon[29849]: osdmap e623: 8 total, 8 up, 8 in 2024-09-11T08:14:47.897 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-126"}]: dispatch 2024-09-11T08:14:48.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:48 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:14:48.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:48 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:14:48.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-126"}]': finished 2024-09-11T08:14:48.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:48 smithi146 ceph-mon[33158]: osdmap e624: 8 total, 8 up, 8 in 2024-09-11T08:14:48.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:48 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:14:48.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:48 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:14:48.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:48 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:14:48.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:48 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:14:48.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:48 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:14:48.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-126"}]': finished 2024-09-11T08:14:48.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:48 smithi102 ceph-mon[41371]: osdmap e624: 8 total, 8 up, 8 in 2024-09-11T08:14:48.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:48 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:14:48.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:48 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:14:48.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:48 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:14:48.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:48 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:14:48.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:48 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:14:48.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-126"}]': finished 2024-09-11T08:14:48.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:48 smithi102 ceph-mon[29849]: osdmap e624: 8 total, 8 up, 8 in 2024-09-11T08:14:48.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:48 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:14:48.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:48 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:14:48.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:48 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:14:49.010 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-09-11T08:14:49.011 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:14:49.011 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9035 ms) 2024-09-11T08:14:49.011 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-09-11T08:14:49.011 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:14:49.011 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6051 ms) 2024-09-11T08:14:49.011 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-09-11T08:14:49.011 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:14:49.011 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5043 ms) 2024-09-11T08:14:49.011 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-09-11T08:14:49.011 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:14:49.011 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (24726 ms) 2024-09-11T08:14:49.012 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-09-11T08:14:49.012 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:14:49.012 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (13626 ms) 2024-09-11T08:14:49.012 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-09-11T08:14:49.012 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12070 ms) 2024-09-11T08:14:49.012 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-09-11T08:14:49.012 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-09-11T08:14:49.012 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: 00000010 69 73 |is| 2024-09-11T08:14:49.012 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: 00000012 2024-09-11T08:14:49.012 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12059 ms) 2024-09-11T08:14:49.012 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-09-11T08:14:49.012 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: require_osd_release = quincy 2024-09-11T08:14:49.012 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6021 ms) 2024-09-11T08:14:49.013 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (554875 ms total) 2024-09-11T08:14:49.013 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: 2024-09-11T08:14:49.013 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-09-11T08:14:49.013 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-09-11T08:14:49.013 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1018 ms) 2024-09-11T08:14:49.013 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-09-11T08:14:49.013 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (11043 ms) 2024-09-11T08:14:49.013 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-09-11T08:14:49.013 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18068 ms) 2024-09-11T08:14:49.013 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-09-11T08:14:49.013 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (2 ms) 2024-09-11T08:14:49.013 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30131 ms total) 2024-09-11T08:14:49.013 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: 2024-09-11T08:14:49.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-09-11T08:14:49.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-09-11T08:14:49.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7054 ms) 2024-09-11T08:14:49.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-09-11T08:14:49.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8048 ms) 2024-09-11T08:14:49.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-09-11T08:14:49.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: waiting for scrub... 2024-09-11T08:14:49.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: done waiting 2024-09-11T08:14:49.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24592 ms) 2024-09-11T08:14:49.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-09-11T08:14:49.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (9378 ms) 2024-09-11T08:14:49.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-09-11T08:14:49.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (7974 ms) 2024-09-11T08:14:49.014 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-09-11T08:14:49.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8055 ms) 2024-09-11T08:14:49.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-09-11T08:14:49.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10071 ms) 2024-09-11T08:14:49.015 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-09-11T08:14:49.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:49 smithi146 ceph-mon[33158]: pgmap v973: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:49.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:49 smithi146 ceph-mon[33158]: osdmap e625: 8 total, 8 up, 8 in 2024-09-11T08:14:49.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:49 smithi102 ceph-mon[41371]: pgmap v973: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:49.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:49 smithi102 ceph-mon[41371]: osdmap e625: 8 total, 8 up, 8 in 2024-09-11T08:14:49.849 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:49 smithi102 ceph-mon[29849]: pgmap v973: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:14:49.849 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:49 smithi102 ceph-mon[29849]: osdmap e625: 8 total, 8 up, 8 in 2024-09-11T08:14:50.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:14:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:14:49] "GET /metrics HTTP/1.1" 200 228690 "" "Prometheus/2.43.0" 2024-09-11T08:14:50.433 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:14:50 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:14:51.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:51 smithi102 ceph-mon[41371]: osdmap e626: 8 total, 8 up, 8 in 2024-09-11T08:14:51.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:51 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:14:51.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:51 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:51.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:51 smithi102 ceph-mon[41371]: pgmap v976: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:14:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:51 smithi102 ceph-mon[29849]: osdmap e626: 8 total, 8 up, 8 in 2024-09-11T08:14:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:51 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:14:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:51 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:51 smithi102 ceph-mon[29849]: pgmap v976: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:14:51.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:51 smithi146 ceph-mon[33158]: osdmap e626: 8 total, 8 up, 8 in 2024-09-11T08:14:51.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:51 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:14:51.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:51 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:14:51.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:51 smithi146 ceph-mon[33158]: pgmap v976: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:14:52.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:52 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:14:52.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:52 smithi102 ceph-mon[41371]: osdmap e627: 8 total, 8 up, 8 in 2024-09-11T08:14:52.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:52 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:14:52.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:52 smithi102 ceph-mon[41371]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:52.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:52 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:14:52.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:52 smithi102 ceph-mon[29849]: osdmap e627: 8 total, 8 up, 8 in 2024-09-11T08:14:52.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:52 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:14:52.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:52 smithi102 ceph-mon[29849]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:52.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:52 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:14:52.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:52 smithi146 ceph-mon[33158]: osdmap e627: 8 total, 8 up, 8 in 2024-09-11T08:14:52.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:52 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:14:52.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:52 smithi146 ceph-mon[33158]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:53.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:53 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:14:53.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:53 smithi102 ceph-mon[41371]: osdmap e628: 8 total, 8 up, 8 in 2024-09-11T08:14:53.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:53 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-128"}]: dispatch 2024-09-11T08:14:53.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:53 smithi102 ceph-mon[41371]: pgmap v979: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:14:53.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:53 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:14:53.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:53 smithi102 ceph-mon[29849]: osdmap e628: 8 total, 8 up, 8 in 2024-09-11T08:14:53.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:53 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-128"}]: dispatch 2024-09-11T08:14:53.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:53 smithi102 ceph-mon[29849]: pgmap v979: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:14:53.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:53 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:14:53.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:53 smithi146 ceph-mon[33158]: osdmap e628: 8 total, 8 up, 8 in 2024-09-11T08:14:53.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:53 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-128"}]: dispatch 2024-09-11T08:14:53.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:53 smithi146 ceph-mon[33158]: pgmap v979: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:14:54.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:54 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-128"}]': finished 2024-09-11T08:14:54.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:54 smithi102 ceph-mon[41371]: osdmap e629: 8 total, 8 up, 8 in 2024-09-11T08:14:54.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:54 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-128", "mode": "writeback"}]: dispatch 2024-09-11T08:14:54.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:54 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:14:54.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:54 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-128", "mode": "writeback"}]': finished 2024-09-11T08:14:54.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:54 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-128"}]': finished 2024-09-11T08:14:54.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:54 smithi102 ceph-mon[29849]: osdmap e629: 8 total, 8 up, 8 in 2024-09-11T08:14:54.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:54 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-128", "mode": "writeback"}]: dispatch 2024-09-11T08:14:54.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:54 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:14:54.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:54 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-128", "mode": "writeback"}]': finished 2024-09-11T08:14:54.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:54 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-128"}]': finished 2024-09-11T08:14:54.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:54 smithi146 ceph-mon[33158]: osdmap e629: 8 total, 8 up, 8 in 2024-09-11T08:14:54.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:54 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-128", "mode": "writeback"}]: dispatch 2024-09-11T08:14:54.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:54 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:14:54.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:54 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-128", "mode": "writeback"}]': finished 2024-09-11T08:14:54.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:14:54 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:14:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:14:55.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:55 smithi102 ceph-mon[41371]: osdmap e630: 8 total, 8 up, 8 in 2024-09-11T08:14:55.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:55 smithi102 ceph-mon[41371]: pgmap v982: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:55.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:55 smithi102 ceph-mon[29849]: osdmap e630: 8 total, 8 up, 8 in 2024-09-11T08:14:55.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:55 smithi102 ceph-mon[29849]: pgmap v982: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:55.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:55 smithi146 ceph-mon[33158]: osdmap e630: 8 total, 8 up, 8 in 2024-09-11T08:14:55.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:55 smithi146 ceph-mon[33158]: pgmap v982: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:14:57.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:57 smithi102 ceph-mon[41371]: pgmap v983: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 368 B/s wr, 1 op/s 2024-09-11T08:14:57.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:57 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:57.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:57 smithi102 ceph-mon[29849]: pgmap v983: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 368 B/s wr, 1 op/s 2024-09-11T08:14:57.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:57 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:57.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:57 smithi146 ceph-mon[33158]: pgmap v983: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 368 B/s wr, 1 op/s 2024-09-11T08:14:57.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:57 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:14:59.895 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:14:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:14:59] "GET /metrics HTTP/1.1" 200 228731 "" "Prometheus/2.43.0" 2024-09-11T08:14:59.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:59 smithi102 ceph-mon[29849]: pgmap v984: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 312 B/s wr, 1 op/s 2024-09-11T08:14:59.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:59 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:14:59.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:59 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:14:59.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:14:59 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:59.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:59 smithi102 ceph-mon[41371]: pgmap v984: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 312 B/s wr, 1 op/s 2024-09-11T08:14:59.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:59 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:14:59.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:59 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:14:59.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:14:59 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:14:59.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:59 smithi146 ceph-mon[33158]: pgmap v984: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 312 B/s wr, 1 op/s 2024-09-11T08:14:59.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:59 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:14:59.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:59 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:14:59.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:14:59 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:15:00.433 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:15:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:15:00.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:00 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:15:00.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:00 smithi102 ceph-mon[29849]: osdmap e631: 8 total, 8 up, 8 in 2024-09-11T08:15:00.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:00 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-128"}]: dispatch 2024-09-11T08:15:00.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:00 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:15:00.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:00 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:15:00.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:00 smithi102 ceph-mon[41371]: osdmap e631: 8 total, 8 up, 8 in 2024-09-11T08:15:00.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:00 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-128"}]: dispatch 2024-09-11T08:15:00.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:00 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:15:00.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:00 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:15:00.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:00 smithi146 ceph-mon[33158]: osdmap e631: 8 total, 8 up, 8 in 2024-09-11T08:15:00.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:00 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-128"}]: dispatch 2024-09-11T08:15:00.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:00 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:15:01.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:01 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:15:01.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:01 smithi102 ceph-mon[29849]: pgmap v986: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 271 B/s wr, 1 op/s 2024-09-11T08:15:01.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:01 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-128"}]': finished 2024-09-11T08:15:01.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:01 smithi102 ceph-mon[29849]: osdmap e632: 8 total, 8 up, 8 in 2024-09-11T08:15:01.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:01 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:15:01.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:01 smithi102 ceph-mon[41371]: pgmap v986: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 271 B/s wr, 1 op/s 2024-09-11T08:15:01.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:01 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-128"}]': finished 2024-09-11T08:15:01.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:01 smithi102 ceph-mon[41371]: osdmap e632: 8 total, 8 up, 8 in 2024-09-11T08:15:01.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:01 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:15:01.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:01 smithi146 ceph-mon[33158]: pgmap v986: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 271 B/s wr, 1 op/s 2024-09-11T08:15:01.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:01 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-128"}]': finished 2024-09-11T08:15:01.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:01 smithi146 ceph-mon[33158]: osdmap e632: 8 total, 8 up, 8 in 2024-09-11T08:15:02.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:02 smithi146 ceph-mon[33158]: osdmap e633: 8 total, 8 up, 8 in 2024-09-11T08:15:03.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:02 smithi102 ceph-mon[29849]: osdmap e633: 8 total, 8 up, 8 in 2024-09-11T08:15:03.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:02 smithi102 ceph-mon[41371]: osdmap e633: 8 total, 8 up, 8 in 2024-09-11T08:15:03.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:03 smithi146 ceph-mon[33158]: pgmap v989: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T08:15:03.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:03 smithi146 ceph-mon[33158]: osdmap e634: 8 total, 8 up, 8 in 2024-09-11T08:15:03.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:03 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:15:04.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:03 smithi102 ceph-mon[41371]: pgmap v989: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T08:15:04.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:03 smithi102 ceph-mon[41371]: osdmap e634: 8 total, 8 up, 8 in 2024-09-11T08:15:04.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:03 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:15:04.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:03 smithi102 ceph-mon[29849]: pgmap v989: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-11T08:15:04.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:03 smithi102 ceph-mon[29849]: osdmap e634: 8 total, 8 up, 8 in 2024-09-11T08:15:04.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:03 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:15:04.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:15:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:15:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:15:04.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:04 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:15:04.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:04 smithi146 ceph-mon[33158]: osdmap e635: 8 total, 8 up, 8 in 2024-09-11T08:15:04.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:04 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:15:05.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:04 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:15:05.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:04 smithi102 ceph-mon[41371]: osdmap e635: 8 total, 8 up, 8 in 2024-09-11T08:15:05.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:04 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:15:05.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:04 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:15:05.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:04 smithi102 ceph-mon[29849]: osdmap e635: 8 total, 8 up, 8 in 2024-09-11T08:15:05.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:04 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:15:06.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:05 smithi102 ceph-mon[41371]: pgmap v992: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:15:06.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:05 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:15:06.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:05 smithi102 ceph-mon[41371]: osdmap e636: 8 total, 8 up, 8 in 2024-09-11T08:15:06.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:05 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-130"}]: dispatch 2024-09-11T08:15:06.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:05 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-130"}]': finished 2024-09-11T08:15:06.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:05 smithi102 ceph-mon[41371]: osdmap e637: 8 total, 8 up, 8 in 2024-09-11T08:15:06.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:05 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-130", "mode": "writeback"}]: dispatch 2024-09-11T08:15:06.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:05 smithi102 ceph-mon[29849]: pgmap v992: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:15:06.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:05 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:15:06.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:05 smithi102 ceph-mon[29849]: osdmap e636: 8 total, 8 up, 8 in 2024-09-11T08:15:06.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:05 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-130"}]: dispatch 2024-09-11T08:15:06.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:05 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-130"}]': finished 2024-09-11T08:15:06.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:05 smithi102 ceph-mon[29849]: osdmap e637: 8 total, 8 up, 8 in 2024-09-11T08:15:06.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:05 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-130", "mode": "writeback"}]: dispatch 2024-09-11T08:15:06.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:05 smithi146 ceph-mon[33158]: pgmap v992: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:15:06.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:05 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:15:06.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:05 smithi146 ceph-mon[33158]: osdmap e636: 8 total, 8 up, 8 in 2024-09-11T08:15:06.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:05 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-130"}]: dispatch 2024-09-11T08:15:06.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:05 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-130"}]': finished 2024-09-11T08:15:06.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:05 smithi146 ceph-mon[33158]: osdmap e637: 8 total, 8 up, 8 in 2024-09-11T08:15:06.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:05 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-130", "mode": "writeback"}]: dispatch 2024-09-11T08:15:07.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:06 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:15:07.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:06 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-130", "mode": "writeback"}]': finished 2024-09-11T08:15:07.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:06 smithi102 ceph-mon[41371]: osdmap e638: 8 total, 8 up, 8 in 2024-09-11T08:15:07.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:06 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:15:07.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:06 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-130", "mode": "writeback"}]': finished 2024-09-11T08:15:07.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:06 smithi102 ceph-mon[29849]: osdmap e638: 8 total, 8 up, 8 in 2024-09-11T08:15:07.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:06 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:15:07.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:06 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-130", "mode": "writeback"}]': finished 2024-09-11T08:15:07.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:06 smithi146 ceph-mon[33158]: osdmap e638: 8 total, 8 up, 8 in 2024-09-11T08:15:08.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:07 smithi102 ceph-mon[41371]: pgmap v995: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:08.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:15:08.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:07 smithi102 ceph-mon[29849]: pgmap v995: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:08.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:15:08.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:07 smithi146 ceph-mon[33158]: pgmap v995: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:08.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:15:09.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:08 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:15:09.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:08 smithi102 ceph-mon[41371]: osdmap e639: 8 total, 8 up, 8 in 2024-09-11T08:15:09.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:08 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-130"}]: dispatch 2024-09-11T08:15:09.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:08 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:15:09.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:08 smithi102 ceph-mon[29849]: osdmap e639: 8 total, 8 up, 8 in 2024-09-11T08:15:09.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:08 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-130"}]: dispatch 2024-09-11T08:15:09.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:08 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:15:09.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:08 smithi146 ceph-mon[33158]: osdmap e639: 8 total, 8 up, 8 in 2024-09-11T08:15:09.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:08 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-130"}]: dispatch 2024-09-11T08:15:10.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:15:09 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:15:09] "GET /metrics HTTP/1.1" 200 228731 "" "Prometheus/2.43.0" 2024-09-11T08:15:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:09 smithi102 ceph-mon[29849]: pgmap v998: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:09 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:15:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:09 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-130"}]': finished 2024-09-11T08:15:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:09 smithi102 ceph-mon[29849]: osdmap e640: 8 total, 8 up, 8 in 2024-09-11T08:15:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:09 smithi102 ceph-mon[41371]: pgmap v998: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:09 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:15:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:09 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-130"}]': finished 2024-09-11T08:15:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:09 smithi102 ceph-mon[41371]: osdmap e640: 8 total, 8 up, 8 in 2024-09-11T08:15:10.433 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:15:10 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:15:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:09 smithi146 ceph-mon[33158]: pgmap v998: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:09 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:15:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:09 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-130"}]': finished 2024-09-11T08:15:10.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:09 smithi146 ceph-mon[33158]: osdmap e640: 8 total, 8 up, 8 in 2024-09-11T08:15:11.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:10 smithi102 ceph-mon[29849]: osdmap e641: 8 total, 8 up, 8 in 2024-09-11T08:15:11.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:10 smithi102 ceph-mon[29849]: osdmap e642: 8 total, 8 up, 8 in 2024-09-11T08:15:11.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:15:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:10 smithi102 ceph-mon[41371]: osdmap e641: 8 total, 8 up, 8 in 2024-09-11T08:15:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:10 smithi102 ceph-mon[41371]: osdmap e642: 8 total, 8 up, 8 in 2024-09-11T08:15:11.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:15:11.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:10 smithi146 ceph-mon[33158]: osdmap e641: 8 total, 8 up, 8 in 2024-09-11T08:15:11.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:10 smithi146 ceph-mon[33158]: osdmap e642: 8 total, 8 up, 8 in 2024-09-11T08:15:11.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:15:12.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:11 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:15:12.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:11 smithi102 ceph-mon[41371]: pgmap v1001: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:15:12.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:11 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:15:12.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:11 smithi102 ceph-mon[41371]: osdmap e643: 8 total, 8 up, 8 in 2024-09-11T08:15:12.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:11 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:15:12.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:11 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:15:12.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:11 smithi102 ceph-mon[29849]: pgmap v1001: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:15:12.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:11 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:15:12.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:11 smithi102 ceph-mon[29849]: osdmap e643: 8 total, 8 up, 8 in 2024-09-11T08:15:12.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:11 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:15:12.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:11 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:15:12.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:11 smithi146 ceph-mon[33158]: pgmap v1001: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:15:12.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:11 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:15:12.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:11 smithi146 ceph-mon[33158]: osdmap e643: 8 total, 8 up, 8 in 2024-09-11T08:15:12.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:11 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:15:14.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:13 smithi102 ceph-mon[29849]: pgmap v1004: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:15:14.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:15:14.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:13 smithi102 ceph-mon[29849]: osdmap e644: 8 total, 8 up, 8 in 2024-09-11T08:15:14.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-132"}]: dispatch 2024-09-11T08:15:14.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:13 smithi102 ceph-mon[41371]: pgmap v1004: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:15:14.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:15:14.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:13 smithi102 ceph-mon[41371]: osdmap e644: 8 total, 8 up, 8 in 2024-09-11T08:15:14.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-132"}]: dispatch 2024-09-11T08:15:14.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:13 smithi146 ceph-mon[33158]: pgmap v1004: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:15:14.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:15:14.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:13 smithi146 ceph-mon[33158]: osdmap e644: 8 total, 8 up, 8 in 2024-09-11T08:15:14.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-132"}]: dispatch 2024-09-11T08:15:14.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:15:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:15:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:15:14.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-132"}]': finished 2024-09-11T08:15:14.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:14 smithi146 ceph-mon[33158]: osdmap e645: 8 total, 8 up, 8 in 2024-09-11T08:15:14.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:14 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-132", "mode": "writeback"}]: dispatch 2024-09-11T08:15:14.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:14 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:15:14.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:14 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:15:14.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:14 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:15:15.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-132"}]': finished 2024-09-11T08:15:15.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:14 smithi102 ceph-mon[41371]: osdmap e645: 8 total, 8 up, 8 in 2024-09-11T08:15:15.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:14 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-132", "mode": "writeback"}]: dispatch 2024-09-11T08:15:15.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:14 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:15:15.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:14 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:15:15.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:14 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:15:15.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-132"}]': finished 2024-09-11T08:15:15.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:14 smithi102 ceph-mon[29849]: osdmap e645: 8 total, 8 up, 8 in 2024-09-11T08:15:15.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:14 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-132", "mode": "writeback"}]: dispatch 2024-09-11T08:15:15.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:14 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:15:15.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:14 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:15:15.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:14 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:15:16.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:15 smithi102 ceph-mon[41371]: pgmap v1007: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:15:16.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:15 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-132", "mode": "writeback"}]': finished 2024-09-11T08:15:16.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:15 smithi102 ceph-mon[41371]: osdmap e646: 8 total, 8 up, 8 in 2024-09-11T08:15:16.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:15 smithi102 ceph-mon[29849]: pgmap v1007: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:15:16.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:15 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-132", "mode": "writeback"}]': finished 2024-09-11T08:15:16.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:15 smithi102 ceph-mon[29849]: osdmap e646: 8 total, 8 up, 8 in 2024-09-11T08:15:16.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:15 smithi146 ceph-mon[33158]: pgmap v1007: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:15:16.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:15 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-132", "mode": "writeback"}]': finished 2024-09-11T08:15:16.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:15 smithi146 ceph-mon[33158]: osdmap e646: 8 total, 8 up, 8 in 2024-09-11T08:15:17.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:16 smithi102 ceph-mon[41371]: osdmap e647: 8 total, 8 up, 8 in 2024-09-11T08:15:17.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:16 smithi102 ceph-mon[41371]: osdmap e648: 8 total, 8 up, 8 in 2024-09-11T08:15:17.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:16 smithi102 ceph-mon[29849]: osdmap e647: 8 total, 8 up, 8 in 2024-09-11T08:15:17.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:16 smithi102 ceph-mon[29849]: osdmap e648: 8 total, 8 up, 8 in 2024-09-11T08:15:17.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:16 smithi146 ceph-mon[33158]: osdmap e647: 8 total, 8 up, 8 in 2024-09-11T08:15:17.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:16 smithi146 ceph-mon[33158]: osdmap e648: 8 total, 8 up, 8 in 2024-09-11T08:15:18.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:17 smithi102 ceph-mon[41371]: pgmap v1010: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:15:18.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:15:18.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:17 smithi102 ceph-mon[29849]: pgmap v1010: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:15:18.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:15:18.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:17 smithi146 ceph-mon[33158]: pgmap v1010: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:15:18.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:15:19.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:15:19.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:18 smithi102 ceph-mon[29849]: osdmap e649: 8 total, 8 up, 8 in 2024-09-11T08:15:19.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-132"}]: dispatch 2024-09-11T08:15:19.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:15:19.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:18 smithi102 ceph-mon[41371]: osdmap e649: 8 total, 8 up, 8 in 2024-09-11T08:15:19.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-132"}]: dispatch 2024-09-11T08:15:19.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:15:19.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:18 smithi146 ceph-mon[33158]: osdmap e649: 8 total, 8 up, 8 in 2024-09-11T08:15:19.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-132"}]: dispatch 2024-09-11T08:15:20.097 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:19 smithi146 ceph-mon[33158]: pgmap v1013: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:15:20.098 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-132"}]': finished 2024-09-11T08:15:20.098 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:19 smithi146 ceph-mon[33158]: osdmap e650: 8 total, 8 up, 8 in 2024-09-11T08:15:20.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:19 smithi102 ceph-mon[29849]: pgmap v1013: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:15:20.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-132"}]': finished 2024-09-11T08:15:20.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:19 smithi102 ceph-mon[29849]: osdmap e650: 8 total, 8 up, 8 in 2024-09-11T08:15:20.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:15:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:15:19] "GET /metrics HTTP/1.1" 200 228698 "" "Prometheus/2.43.0" 2024-09-11T08:15:20.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:19 smithi102 ceph-mon[41371]: pgmap v1013: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:15:20.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-132"}]': finished 2024-09-11T08:15:20.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:19 smithi102 ceph-mon[41371]: osdmap e650: 8 total, 8 up, 8 in 2024-09-11T08:15:20.433 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:15:20 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:15:21.047 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:20 smithi102 ceph-mon[41371]: osdmap e651: 8 total, 8 up, 8 in 2024-09-11T08:15:21.047 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:15:21.047 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:20 smithi102 ceph-mon[29849]: osdmap e651: 8 total, 8 up, 8 in 2024-09-11T08:15:21.047 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:15:21.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:20 smithi146 ceph-mon[33158]: osdmap e651: 8 total, 8 up, 8 in 2024-09-11T08:15:21.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:15:22.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:21 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:15:22.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:21 smithi102 ceph-mon[29849]: pgmap v1016: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-11T08:15:22.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:21 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:15:22.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:21 smithi102 ceph-mon[29849]: osdmap e652: 8 total, 8 up, 8 in 2024-09-11T08:15:22.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:21 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-132"}]: dispatch 2024-09-11T08:15:22.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:21 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:15:22.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:21 smithi102 ceph-mon[41371]: pgmap v1016: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-11T08:15:22.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:21 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:15:22.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:21 smithi102 ceph-mon[41371]: osdmap e652: 8 total, 8 up, 8 in 2024-09-11T08:15:22.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:21 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-132"}]: dispatch 2024-09-11T08:15:22.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:21 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:15:22.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:21 smithi146 ceph-mon[33158]: pgmap v1016: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-11T08:15:22.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:21 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:15:22.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:21 smithi146 ceph-mon[33158]: osdmap e652: 8 total, 8 up, 8 in 2024-09-11T08:15:22.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:21 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-132"}]: dispatch 2024-09-11T08:15:23.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:22 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:15:23.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-132"}]': finished 2024-09-11T08:15:23.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:22 smithi102 ceph-mon[41371]: osdmap e653: 8 total, 8 up, 8 in 2024-09-11T08:15:23.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:22 smithi102 ceph-mon[41371]: osdmap e654: 8 total, 8 up, 8 in 2024-09-11T08:15:23.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:22 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:15:23.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-132"}]': finished 2024-09-11T08:15:23.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:22 smithi102 ceph-mon[29849]: osdmap e653: 8 total, 8 up, 8 in 2024-09-11T08:15:23.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:22 smithi102 ceph-mon[29849]: osdmap e654: 8 total, 8 up, 8 in 2024-09-11T08:15:23.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:22 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:15:23.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-132"}]': finished 2024-09-11T08:15:23.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:22 smithi146 ceph-mon[33158]: osdmap e653: 8 total, 8 up, 8 in 2024-09-11T08:15:23.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:22 smithi146 ceph-mon[33158]: osdmap e654: 8 total, 8 up, 8 in 2024-09-11T08:15:24.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:24 smithi102 ceph-mon[41371]: pgmap v1020: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 536 B/s wr, 2 op/s 2024-09-11T08:15:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:24 smithi102 ceph-mon[41371]: osdmap e655: 8 total, 8 up, 8 in 2024-09-11T08:15:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:15:24.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:24 smithi102 ceph-mon[29849]: pgmap v1020: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 536 B/s wr, 2 op/s 2024-09-11T08:15:24.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:24 smithi102 ceph-mon[29849]: osdmap e655: 8 total, 8 up, 8 in 2024-09-11T08:15:24.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:15:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:24 smithi146 ceph-mon[33158]: pgmap v1020: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 536 B/s wr, 2 op/s 2024-09-11T08:15:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:24 smithi146 ceph-mon[33158]: osdmap e655: 8 total, 8 up, 8 in 2024-09-11T08:15:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:15:24.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:15:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:15:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:15:25.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:15:25.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:25 smithi102 ceph-mon[41371]: osdmap e656: 8 total, 8 up, 8 in 2024-09-11T08:15:25.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:15:25.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:25 smithi102 ceph-mon[41371]: pgmap v1023: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:15:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:15:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:25 smithi102 ceph-mon[29849]: osdmap e656: 8 total, 8 up, 8 in 2024-09-11T08:15:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:15:25.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:25 smithi102 ceph-mon[29849]: pgmap v1023: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:15:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:15:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:25 smithi146 ceph-mon[33158]: osdmap e656: 8 total, 8 up, 8 in 2024-09-11T08:15:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:15:25.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:25 smithi146 ceph-mon[33158]: pgmap v1023: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:15:26.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:15:26.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:26 smithi102 ceph-mon[29849]: osdmap e657: 8 total, 8 up, 8 in 2024-09-11T08:15:26.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-134"}]: dispatch 2024-09-11T08:15:26.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:15:26.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:26 smithi102 ceph-mon[41371]: osdmap e657: 8 total, 8 up, 8 in 2024-09-11T08:15:26.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-134"}]: dispatch 2024-09-11T08:15:26.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:15:26.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:26 smithi146 ceph-mon[33158]: osdmap e657: 8 total, 8 up, 8 in 2024-09-11T08:15:26.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-134"}]: dispatch 2024-09-11T08:15:27.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-134"}]': finished 2024-09-11T08:15:27.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:27 smithi102 ceph-mon[41371]: osdmap e658: 8 total, 8 up, 8 in 2024-09-11T08:15:27.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:27 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-134", "mode": "writeback"}]: dispatch 2024-09-11T08:15:27.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:27 smithi102 ceph-mon[41371]: pgmap v1026: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:27.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:27 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:15:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-134"}]': finished 2024-09-11T08:15:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:27 smithi102 ceph-mon[29849]: osdmap e658: 8 total, 8 up, 8 in 2024-09-11T08:15:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:27 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-134", "mode": "writeback"}]: dispatch 2024-09-11T08:15:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:27 smithi102 ceph-mon[29849]: pgmap v1026: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:27.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:27 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:15:27.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-134"}]': finished 2024-09-11T08:15:27.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:27 smithi146 ceph-mon[33158]: osdmap e658: 8 total, 8 up, 8 in 2024-09-11T08:15:27.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:27 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-134", "mode": "writeback"}]: dispatch 2024-09-11T08:15:27.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:27 smithi146 ceph-mon[33158]: pgmap v1026: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:27.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:27 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:15:28.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-134", "mode": "writeback"}]': finished 2024-09-11T08:15:28.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:28 smithi102 ceph-mon[41371]: osdmap e659: 8 total, 8 up, 8 in 2024-09-11T08:15:28.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:28 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:15:28.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-134", "mode": "writeback"}]': finished 2024-09-11T08:15:28.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:28 smithi102 ceph-mon[29849]: osdmap e659: 8 total, 8 up, 8 in 2024-09-11T08:15:28.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:28 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:15:28.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-134", "mode": "writeback"}]': finished 2024-09-11T08:15:28.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:28 smithi146 ceph-mon[33158]: osdmap e659: 8 total, 8 up, 8 in 2024-09-11T08:15:28.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:28 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:15:29.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:15:29.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:29 smithi102 ceph-mon[29849]: osdmap e660: 8 total, 8 up, 8 in 2024-09-11T08:15:29.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-134"}]: dispatch 2024-09-11T08:15:29.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:29 smithi102 ceph-mon[29849]: pgmap v1029: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:29.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:29 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:15:29.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:15:29.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:29 smithi102 ceph-mon[41371]: osdmap e660: 8 total, 8 up, 8 in 2024-09-11T08:15:29.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-134"}]: dispatch 2024-09-11T08:15:29.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:29 smithi102 ceph-mon[41371]: pgmap v1029: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:29.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:29 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:15:29.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:15:29.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:29 smithi146 ceph-mon[33158]: osdmap e660: 8 total, 8 up, 8 in 2024-09-11T08:15:29.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-134"}]: dispatch 2024-09-11T08:15:29.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:29 smithi146 ceph-mon[33158]: pgmap v1029: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:29.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:29 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:15:30.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:15:29 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:15:29] "GET /metrics HTTP/1.1" 200 228730 "" "Prometheus/2.43.0" 2024-09-11T08:15:30.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:30 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-134"}]': finished 2024-09-11T08:15:30.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:30 smithi102 ceph-mon[41371]: osdmap e661: 8 total, 8 up, 8 in 2024-09-11T08:15:30.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:30 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:15:30.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:30 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-134"}]': finished 2024-09-11T08:15:30.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:30 smithi102 ceph-mon[29849]: osdmap e661: 8 total, 8 up, 8 in 2024-09-11T08:15:30.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:30 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:15:30.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:30 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-134"}]': finished 2024-09-11T08:15:30.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:30 smithi146 ceph-mon[33158]: osdmap e661: 8 total, 8 up, 8 in 2024-09-11T08:15:30.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:30 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:15:30.434 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:15:30 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:15:31.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:31 smithi102 ceph-mon[41371]: osdmap e662: 8 total, 8 up, 8 in 2024-09-11T08:15:31.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:31 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:15:31.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:31 smithi102 ceph-mon[41371]: pgmap v1032: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:15:31.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:31 smithi102 ceph-mon[29849]: osdmap e662: 8 total, 8 up, 8 in 2024-09-11T08:15:31.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:31 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:15:31.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:31 smithi102 ceph-mon[29849]: pgmap v1032: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:15:31.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:31 smithi146 ceph-mon[33158]: osdmap e662: 8 total, 8 up, 8 in 2024-09-11T08:15:31.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:31 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:15:31.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:31 smithi146 ceph-mon[33158]: pgmap v1032: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:15:32.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:32 smithi102 ceph-mon[41371]: osdmap e663: 8 total, 8 up, 8 in 2024-09-11T08:15:32.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:15:32.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:32 smithi102 ceph-mon[29849]: osdmap e663: 8 total, 8 up, 8 in 2024-09-11T08:15:32.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:15:32.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:32 smithi146 ceph-mon[33158]: osdmap e663: 8 total, 8 up, 8 in 2024-09-11T08:15:32.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:15:33.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:15:33.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:33 smithi102 ceph-mon[29849]: osdmap e664: 8 total, 8 up, 8 in 2024-09-11T08:15:33.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:15:33.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:33 smithi102 ceph-mon[29849]: pgmap v1035: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:15:33.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:15:33.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:33 smithi102 ceph-mon[29849]: osdmap e665: 8 total, 8 up, 8 in 2024-09-11T08:15:33.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-136"}]: dispatch 2024-09-11T08:15:33.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:15:33.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:33 smithi102 ceph-mon[41371]: osdmap e664: 8 total, 8 up, 8 in 2024-09-11T08:15:33.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:15:33.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:33 smithi102 ceph-mon[41371]: pgmap v1035: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:15:33.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:15:33.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:33 smithi102 ceph-mon[41371]: osdmap e665: 8 total, 8 up, 8 in 2024-09-11T08:15:33.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-136"}]: dispatch 2024-09-11T08:15:33.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:15:33.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:33 smithi146 ceph-mon[33158]: osdmap e664: 8 total, 8 up, 8 in 2024-09-11T08:15:33.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:15:33.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:33 smithi146 ceph-mon[33158]: pgmap v1035: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-11T08:15:33.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:15:33.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:33 smithi146 ceph-mon[33158]: osdmap e665: 8 total, 8 up, 8 in 2024-09-11T08:15:33.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-136"}]: dispatch 2024-09-11T08:15:34.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:15:34 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:15:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:15:35.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-136"}]': finished 2024-09-11T08:15:35.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:35 smithi102 ceph-mon[29849]: osdmap e666: 8 total, 8 up, 8 in 2024-09-11T08:15:35.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-136", "mode": "writeback"}]: dispatch 2024-09-11T08:15:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:35 smithi102 ceph-mon[29849]: pgmap v1038: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:15:35.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-136"}]': finished 2024-09-11T08:15:35.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:35 smithi102 ceph-mon[41371]: osdmap e666: 8 total, 8 up, 8 in 2024-09-11T08:15:35.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-136", "mode": "writeback"}]: dispatch 2024-09-11T08:15:35.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:35 smithi102 ceph-mon[41371]: pgmap v1038: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:15:35.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-136"}]': finished 2024-09-11T08:15:35.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:35 smithi146 ceph-mon[33158]: osdmap e666: 8 total, 8 up, 8 in 2024-09-11T08:15:35.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-136", "mode": "writeback"}]: dispatch 2024-09-11T08:15:35.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:35 smithi146 ceph-mon[33158]: pgmap v1038: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:15:36.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:36 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:15:36.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:36 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-136", "mode": "writeback"}]': finished 2024-09-11T08:15:36.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:36 smithi102 ceph-mon[29849]: osdmap e667: 8 total, 8 up, 8 in 2024-09-11T08:15:36.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:36 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:15:36.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:36 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:15:36.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:36 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-136", "mode": "writeback"}]': finished 2024-09-11T08:15:36.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:36 smithi102 ceph-mon[41371]: osdmap e667: 8 total, 8 up, 8 in 2024-09-11T08:15:36.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:36 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:15:36.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:36 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:15:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:36 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-136", "mode": "writeback"}]': finished 2024-09-11T08:15:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:36 smithi146 ceph-mon[33158]: osdmap e667: 8 total, 8 up, 8 in 2024-09-11T08:15:36.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:36 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:15:37.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:15:37.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:37 smithi102 ceph-mon[29849]: osdmap e668: 8 total, 8 up, 8 in 2024-09-11T08:15:37.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-136"}]: dispatch 2024-09-11T08:15:37.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:37 smithi102 ceph-mon[29849]: pgmap v1041: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-09-11T08:15:37.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:37 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:15:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:15:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:37 smithi102 ceph-mon[41371]: osdmap e668: 8 total, 8 up, 8 in 2024-09-11T08:15:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-136"}]: dispatch 2024-09-11T08:15:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:37 smithi102 ceph-mon[41371]: pgmap v1041: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-09-11T08:15:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:37 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:15:37.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:15:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:37 smithi146 ceph-mon[33158]: osdmap e668: 8 total, 8 up, 8 in 2024-09-11T08:15:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-136"}]: dispatch 2024-09-11T08:15:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:37 smithi146 ceph-mon[33158]: pgmap v1041: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-09-11T08:15:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:37 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:15:38.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-136"}]': finished 2024-09-11T08:15:38.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:38 smithi102 ceph-mon[29849]: osdmap e669: 8 total, 8 up, 8 in 2024-09-11T08:15:38.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-136"}]': finished 2024-09-11T08:15:38.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:38 smithi102 ceph-mon[41371]: osdmap e669: 8 total, 8 up, 8 in 2024-09-11T08:15:38.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-136"}]': finished 2024-09-11T08:15:38.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:38 smithi146 ceph-mon[33158]: osdmap e669: 8 total, 8 up, 8 in 2024-09-11T08:15:39.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:39 smithi102 ceph-mon[41371]: osdmap e670: 8 total, 8 up, 8 in 2024-09-11T08:15:39.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:39 smithi102 ceph-mon[41371]: pgmap v1044: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-11T08:15:39.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:39 smithi102 ceph-mon[41371]: osdmap e671: 8 total, 8 up, 8 in 2024-09-11T08:15:39.647 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:39 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:15:39.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:39 smithi102 ceph-mon[29849]: osdmap e670: 8 total, 8 up, 8 in 2024-09-11T08:15:39.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:39 smithi102 ceph-mon[29849]: pgmap v1044: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-11T08:15:39.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:39 smithi102 ceph-mon[29849]: osdmap e671: 8 total, 8 up, 8 in 2024-09-11T08:15:39.648 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:39 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:15:39.687 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:39 smithi146 ceph-mon[33158]: osdmap e670: 8 total, 8 up, 8 in 2024-09-11T08:15:39.687 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:39 smithi146 ceph-mon[33158]: pgmap v1044: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-11T08:15:39.687 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:39 smithi146 ceph-mon[33158]: osdmap e671: 8 total, 8 up, 8 in 2024-09-11T08:15:39.687 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:39 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:15:40.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:15:39 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:15:39] "GET /metrics HTTP/1.1" 200 228730 "" "Prometheus/2.43.0" 2024-09-11T08:15:40.433 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:15:40 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:15:41.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:15:41.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:41 smithi102 ceph-mon[29849]: osdmap e672: 8 total, 8 up, 8 in 2024-09-11T08:15:41.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:15:41.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:41 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:15:41.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:41 smithi102 ceph-mon[29849]: pgmap v1047: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:41.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:15:41.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:41 smithi102 ceph-mon[41371]: osdmap e672: 8 total, 8 up, 8 in 2024-09-11T08:15:41.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:15:41.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:41 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:15:41.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:41 smithi102 ceph-mon[41371]: pgmap v1047: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:41.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:15:41.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:41 smithi146 ceph-mon[33158]: osdmap e672: 8 total, 8 up, 8 in 2024-09-11T08:15:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:15:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:41 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:15:41.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:41 smithi146 ceph-mon[33158]: pgmap v1047: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:42.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:15:42.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:42 smithi102 ceph-mon[41371]: osdmap e673: 8 total, 8 up, 8 in 2024-09-11T08:15:42.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-138","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:15:42.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:15:42.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:42 smithi102 ceph-mon[29849]: osdmap e673: 8 total, 8 up, 8 in 2024-09-11T08:15:42.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-138","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:15:42.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:15:42.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:42 smithi146 ceph-mon[33158]: osdmap e673: 8 total, 8 up, 8 in 2024-09-11T08:15:42.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-138","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:15:43.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:43 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-138","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:15:43.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:43 smithi102 ceph-mon[41371]: osdmap e674: 8 total, 8 up, 8 in 2024-09-11T08:15:43.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:43 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-138","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:15:43.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:43 smithi102 ceph-mon[41371]: pgmap v1050: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:43.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:43 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-138","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:15:43.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:43 smithi102 ceph-mon[41371]: osdmap e675: 8 total, 8 up, 8 in 2024-09-11T08:15:43.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:43 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:15:43.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:43 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-138","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:15:43.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:43 smithi102 ceph-mon[29849]: osdmap e674: 8 total, 8 up, 8 in 2024-09-11T08:15:43.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:43 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-138","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:15:43.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:43 smithi102 ceph-mon[29849]: pgmap v1050: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:43.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:43 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-138","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:15:43.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:43 smithi102 ceph-mon[29849]: osdmap e675: 8 total, 8 up, 8 in 2024-09-11T08:15:43.397 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:43 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:15:43.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:43 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-138","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:15:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:43 smithi146 ceph-mon[33158]: osdmap e674: 8 total, 8 up, 8 in 2024-09-11T08:15:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:43 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-138","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:15:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:43 smithi146 ceph-mon[33158]: pgmap v1050: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:43 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-138","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:15:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:43 smithi146 ceph-mon[33158]: osdmap e675: 8 total, 8 up, 8 in 2024-09-11T08:15:43.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:43 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-11T08:15:44.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:15:44 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:15:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:15:45.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:45 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:15:45.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:45 smithi102 ceph-mon[41371]: osdmap e676: 8 total, 8 up, 8 in 2024-09-11T08:15:45.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:45 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:15:45.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:45 smithi102 ceph-mon[41371]: pgmap v1053: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:15:45.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:45 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:15:45.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:45 smithi102 ceph-mon[29849]: osdmap e676: 8 total, 8 up, 8 in 2024-09-11T08:15:45.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:45 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:15:45.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:45 smithi102 ceph-mon[29849]: pgmap v1053: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:15:45.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:45 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-11T08:15:45.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:45 smithi146 ceph-mon[33158]: osdmap e676: 8 total, 8 up, 8 in 2024-09-11T08:15:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:45 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:15:45.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:45 smithi146 ceph-mon[33158]: pgmap v1053: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:15:46.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:46 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:15:46.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:46 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-138"}]: dispatch 2024-09-11T08:15:46.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:15:46.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-138"}]: dispatch 2024-09-11T08:15:46.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:46 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:15:46.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:46 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-138"}]: dispatch 2024-09-11T08:15:47.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-138"}]': finished 2024-09-11T08:15:47.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:47 smithi102 ceph-mon[41371]: osdmap e677: 8 total, 8 up, 8 in 2024-09-11T08:15:47.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:47 smithi102 ceph-mon[41371]: pgmap v1055: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:15:47.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-138"}]': finished 2024-09-11T08:15:47.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:47 smithi102 ceph-mon[29849]: osdmap e677: 8 total, 8 up, 8 in 2024-09-11T08:15:47.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:47 smithi102 ceph-mon[29849]: pgmap v1055: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:15:47.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-138"}]': finished 2024-09-11T08:15:47.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:47 smithi146 ceph-mon[33158]: osdmap e677: 8 total, 8 up, 8 in 2024-09-11T08:15:47.685 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:47 smithi146 ceph-mon[33158]: pgmap v1055: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:15:48.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:48 smithi102 ceph-mon[41371]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:15:48.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:48 smithi102 ceph-mon[41371]: osdmap e678: 8 total, 8 up, 8 in 2024-09-11T08:15:48.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:48 smithi102 ceph-mon[41371]: osdmap e679: 8 total, 8 up, 8 in 2024-09-11T08:15:48.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:48 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:15:48.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:48 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:15:48.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:48 smithi102 ceph-mon[29849]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:15:48.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:48 smithi102 ceph-mon[29849]: osdmap e678: 8 total, 8 up, 8 in 2024-09-11T08:15:48.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:48 smithi102 ceph-mon[29849]: osdmap e679: 8 total, 8 up, 8 in 2024-09-11T08:15:48.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:48 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:15:48.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:48 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:15:48.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:48 smithi146 ceph-mon[33158]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:15:48.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:48 smithi146 ceph-mon[33158]: osdmap e678: 8 total, 8 up, 8 in 2024-09-11T08:15:48.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:48 smithi146 ceph-mon[33158]: osdmap e679: 8 total, 8 up, 8 in 2024-09-11T08:15:48.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:48 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:15:48.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:48 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:15:49.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:49 smithi102 ceph-mon[29849]: pgmap v1058: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:15:49.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:49 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:15:49.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:49 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:15:49.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:49 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:15:49.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:49 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:15:49.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:49 smithi102 ceph-mon[29849]: osdmap e680: 8 total, 8 up, 8 in 2024-09-11T08:15:49.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:49 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:15:49.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:49 smithi102 ceph-mon[41371]: pgmap v1058: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:15:49.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:49 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:15:49.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:49 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:15:49.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:49 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:15:49.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:49 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:15:49.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:49 smithi102 ceph-mon[41371]: osdmap e680: 8 total, 8 up, 8 in 2024-09-11T08:15:49.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:49 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:15:49.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:49 smithi146 ceph-mon[33158]: pgmap v1058: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:15:49.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:49 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:15:49.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:49 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:15:49.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:49 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:15:49.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:49 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:15:49.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:49 smithi146 ceph-mon[33158]: osdmap e680: 8 total, 8 up, 8 in 2024-09-11T08:15:49.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:49 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:15:50.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:15:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:15:49] "GET /metrics HTTP/1.1" 200 228731 "" "Prometheus/2.43.0" 2024-09-11T08:15:50.433 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:15:50 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:15:51.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:51 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:15:51.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:51 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:15:51.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:51 smithi102 ceph-mon[29849]: osdmap e681: 8 total, 8 up, 8 in 2024-09-11T08:15:51.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:51 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_count","val": "3"}]: dispatch 2024-09-11T08:15:51.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:51 smithi102 ceph-mon[29849]: pgmap v1061: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:51.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:51 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:15:51.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:51 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:15:51.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:51 smithi102 ceph-mon[41371]: osdmap e681: 8 total, 8 up, 8 in 2024-09-11T08:15:51.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:51 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_count","val": "3"}]: dispatch 2024-09-11T08:15:51.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:51 smithi102 ceph-mon[41371]: pgmap v1061: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:51.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:51 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:15:51.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:51 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:15:51.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:51 smithi146 ceph-mon[33158]: osdmap e681: 8 total, 8 up, 8 in 2024-09-11T08:15:51.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:51 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_count","val": "3"}]: dispatch 2024-09-11T08:15:51.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:51 smithi146 ceph-mon[33158]: pgmap v1061: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:52.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:52 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_count","val": "3"}]': finished 2024-09-11T08:15:52.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:52 smithi146 ceph-mon[33158]: osdmap e682: 8 total, 8 up, 8 in 2024-09-11T08:15:52.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:52 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_period","val": "3"}]: dispatch 2024-09-11T08:15:52.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:52 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_count","val": "3"}]': finished 2024-09-11T08:15:52.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:52 smithi102 ceph-mon[29849]: osdmap e682: 8 total, 8 up, 8 in 2024-09-11T08:15:52.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:52 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_period","val": "3"}]: dispatch 2024-09-11T08:15:52.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:52 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_count","val": "3"}]': finished 2024-09-11T08:15:52.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:52 smithi102 ceph-mon[41371]: osdmap e682: 8 total, 8 up, 8 in 2024-09-11T08:15:52.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:52 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_period","val": "3"}]: dispatch 2024-09-11T08:15:53.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:53 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_period","val": "3"}]': finished 2024-09-11T08:15:53.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:53 smithi146 ceph-mon[33158]: osdmap e683: 8 total, 8 up, 8 in 2024-09-11T08:15:53.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:53 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:15:53.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:53 smithi146 ceph-mon[33158]: pgmap v1064: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:53.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:53 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:15:53.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:53 smithi146 ceph-mon[33158]: osdmap e684: 8 total, 8 up, 8 in 2024-09-11T08:15:53.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:53 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_period","val": "3"}]': finished 2024-09-11T08:15:53.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:53 smithi102 ceph-mon[41371]: osdmap e683: 8 total, 8 up, 8 in 2024-09-11T08:15:53.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:53 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:15:53.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:53 smithi102 ceph-mon[41371]: pgmap v1064: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:53.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:53 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:15:53.646 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:53 smithi102 ceph-mon[41371]: osdmap e684: 8 total, 8 up, 8 in 2024-09-11T08:15:53.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:53 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_period","val": "3"}]': finished 2024-09-11T08:15:53.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:53 smithi102 ceph-mon[29849]: osdmap e683: 8 total, 8 up, 8 in 2024-09-11T08:15:53.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:53 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:15:53.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:53 smithi102 ceph-mon[29849]: pgmap v1064: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:15:53.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:53 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:15:53.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:53 smithi102 ceph-mon[29849]: osdmap e684: 8 total, 8 up, 8 in 2024-09-11T08:15:54.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:54 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-11T08:15:54.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:54 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-11T08:15:54.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:54 smithi146 ceph-mon[33158]: osdmap e685: 8 total, 8 up, 8 in 2024-09-11T08:15:54.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:54 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-11T08:15:54.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:54 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-11T08:15:54.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:54 smithi102 ceph-mon[41371]: osdmap e685: 8 total, 8 up, 8 in 2024-09-11T08:15:54.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:54 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-11T08:15:54.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:54 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-11T08:15:54.646 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:54 smithi102 ceph-mon[29849]: osdmap e685: 8 total, 8 up, 8 in 2024-09-11T08:15:54.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:15:54 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:15:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:15:55.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:55 smithi146 ceph-mon[33158]: pgmap v1067: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:15:55.645 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:55 smithi102 ceph-mon[29849]: pgmap v1067: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:15:55.645 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:55 smithi102 ceph-mon[41371]: pgmap v1067: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:15:57.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:57 smithi146 ceph-mon[33158]: pgmap v1068: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 2.9 KiB/s wr, 1 op/s 2024-09-11T08:15:57.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:57 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:15:58.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:57 smithi102 ceph-mon[41371]: pgmap v1068: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 2.9 KiB/s wr, 1 op/s 2024-09-11T08:15:58.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:57 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:15:58.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:57 smithi102 ceph-mon[29849]: pgmap v1068: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 2.9 KiB/s wr, 1 op/s 2024-09-11T08:15:58.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:57 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:15:59.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:59 smithi146 ceph-mon[33158]: pgmap v1069: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 793 B/s rd, 2.5 KiB/s wr, 1 op/s 2024-09-11T08:15:59.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:15:59 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:16:00.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:59 smithi102 ceph-mon[29849]: pgmap v1069: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 793 B/s rd, 2.5 KiB/s wr, 1 op/s 2024-09-11T08:16:00.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:15:59 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:16:00.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:59 smithi102 ceph-mon[41371]: pgmap v1069: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 793 B/s rd, 2.5 KiB/s wr, 1 op/s 2024-09-11T08:16:00.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:15:59 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:16:00.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:15:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:15:59] "GET /metrics HTTP/1.1" 200 228764 "" "Prometheus/2.43.0" 2024-09-11T08:16:00.433 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:16:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:16:01.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:01 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:16:01.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:01 smithi146 ceph-mon[33158]: pgmap v1070: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.2 KiB/s wr, 2 op/s 2024-09-11T08:16:02.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:01 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:16:02.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:01 smithi102 ceph-mon[29849]: pgmap v1070: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.2 KiB/s wr, 2 op/s 2024-09-11T08:16:02.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:01 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:16:02.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:01 smithi102 ceph-mon[41371]: pgmap v1070: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.2 KiB/s wr, 2 op/s 2024-09-11T08:16:03.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:03 smithi146 ceph-mon[33158]: pgmap v1071: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 6.1 KiB/s wr, 2 op/s 2024-09-11T08:16:04.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:03 smithi102 ceph-mon[29849]: pgmap v1071: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 6.1 KiB/s wr, 2 op/s 2024-09-11T08:16:04.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:03 smithi102 ceph-mon[41371]: pgmap v1071: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 6.1 KiB/s wr, 2 op/s 2024-09-11T08:16:04.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:16:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:16:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:16:05.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:05 smithi146 ceph-mon[33158]: pgmap v1072: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 979 B/s rd, 5.5 KiB/s wr, 1 op/s 2024-09-11T08:16:06.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:05 smithi102 ceph-mon[29849]: pgmap v1072: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 979 B/s rd, 5.5 KiB/s wr, 1 op/s 2024-09-11T08:16:06.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:05 smithi102 ceph-mon[41371]: pgmap v1072: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 979 B/s rd, 5.5 KiB/s wr, 1 op/s 2024-09-11T08:16:06.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:06 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:16:06.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:06 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-140"}]: dispatch 2024-09-11T08:16:07.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:06 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:16:07.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:06 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-140"}]: dispatch 2024-09-11T08:16:07.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:06 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:16:07.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:06 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-140"}]: dispatch 2024-09-11T08:16:08.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:07 smithi146 ceph-mon[33158]: pgmap v1073: 204 pgs: 204 active+clean; 4.5 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-09-11T08:16:08.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-140"}]': finished 2024-09-11T08:16:08.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:07 smithi146 ceph-mon[33158]: osdmap e686: 8 total, 8 up, 8 in 2024-09-11T08:16:08.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:07 smithi102 ceph-mon[29849]: pgmap v1073: 204 pgs: 204 active+clean; 4.5 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-09-11T08:16:08.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-140"}]': finished 2024-09-11T08:16:08.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:07 smithi102 ceph-mon[29849]: osdmap e686: 8 total, 8 up, 8 in 2024-09-11T08:16:08.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:07 smithi102 ceph-mon[41371]: pgmap v1073: 204 pgs: 204 active+clean; 4.5 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 8.2 KiB/s wr, 2 op/s 2024-09-11T08:16:08.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-140"}]': finished 2024-09-11T08:16:08.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:07 smithi102 ceph-mon[41371]: osdmap e686: 8 total, 8 up, 8 in 2024-09-11T08:16:09.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:08 smithi146 ceph-mon[33158]: osdmap e687: 8 total, 8 up, 8 in 2024-09-11T08:16:09.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:08 smithi146 ceph-mon[33158]: osdmap e688: 8 total, 8 up, 8 in 2024-09-11T08:16:09.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:08 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:16:09.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:08 smithi102 ceph-mon[29849]: osdmap e687: 8 total, 8 up, 8 in 2024-09-11T08:16:09.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:08 smithi102 ceph-mon[29849]: osdmap e688: 8 total, 8 up, 8 in 2024-09-11T08:16:09.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:08 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:16:09.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:08 smithi102 ceph-mon[41371]: osdmap e687: 8 total, 8 up, 8 in 2024-09-11T08:16:09.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:08 smithi102 ceph-mon[41371]: osdmap e688: 8 total, 8 up, 8 in 2024-09-11T08:16:09.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:08 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:16:10.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:16:09 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:16:09] "GET /metrics HTTP/1.1" 200 228764 "" "Prometheus/2.43.0" 2024-09-11T08:16:10.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:09 smithi102 ceph-mon[41371]: pgmap v1076: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:16:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:09 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:16:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:09 smithi102 ceph-mon[41371]: osdmap e689: 8 total, 8 up, 8 in 2024-09-11T08:16:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:09 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:16:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:09 smithi102 ceph-mon[29849]: pgmap v1076: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:16:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:09 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:16:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:09 smithi102 ceph-mon[29849]: osdmap e689: 8 total, 8 up, 8 in 2024-09-11T08:16:10.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:09 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:16:10.183 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:16:10 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:16:10.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:09 smithi146 ceph-mon[33158]: pgmap v1076: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T08:16:10.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:09 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:16:10.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:09 smithi146 ceph-mon[33158]: osdmap e689: 8 total, 8 up, 8 in 2024-09-11T08:16:10.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:09 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:16:11.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:11 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:16:11.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:11 smithi146 ceph-mon[33158]: pgmap v1079: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-11T08:16:11.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:11 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:16:11.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:11 smithi146 ceph-mon[33158]: osdmap e690: 8 total, 8 up, 8 in 2024-09-11T08:16:11.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:11 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-142"}]: dispatch 2024-09-11T08:16:12.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:11 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:16:12.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:11 smithi102 ceph-mon[29849]: pgmap v1079: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-11T08:16:12.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:11 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:16:12.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:11 smithi102 ceph-mon[29849]: osdmap e690: 8 total, 8 up, 8 in 2024-09-11T08:16:12.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:11 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-142"}]: dispatch 2024-09-11T08:16:12.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:11 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:16:12.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:11 smithi102 ceph-mon[41371]: pgmap v1079: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-11T08:16:12.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:11 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:16:12.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:11 smithi102 ceph-mon[41371]: osdmap e690: 8 total, 8 up, 8 in 2024-09-11T08:16:12.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:11 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-142"}]: dispatch 2024-09-11T08:16:12.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-142"}]': finished 2024-09-11T08:16:12.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:12 smithi146 ceph-mon[33158]: osdmap e691: 8 total, 8 up, 8 in 2024-09-11T08:16:12.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-142", "mode": "writeback"}]: dispatch 2024-09-11T08:16:12.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:12 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:16:12.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-142", "mode": "writeback"}]': finished 2024-09-11T08:16:12.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:12 smithi146 ceph-mon[33158]: osdmap e692: 8 total, 8 up, 8 in 2024-09-11T08:16:12.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:16:13.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-142"}]': finished 2024-09-11T08:16:13.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:12 smithi102 ceph-mon[41371]: osdmap e691: 8 total, 8 up, 8 in 2024-09-11T08:16:13.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-142", "mode": "writeback"}]: dispatch 2024-09-11T08:16:13.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:12 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:16:13.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-142", "mode": "writeback"}]': finished 2024-09-11T08:16:13.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:12 smithi102 ceph-mon[41371]: osdmap e692: 8 total, 8 up, 8 in 2024-09-11T08:16:13.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:16:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-142"}]': finished 2024-09-11T08:16:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:12 smithi102 ceph-mon[29849]: osdmap e691: 8 total, 8 up, 8 in 2024-09-11T08:16:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-142", "mode": "writeback"}]: dispatch 2024-09-11T08:16:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:12 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:16:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-142", "mode": "writeback"}]': finished 2024-09-11T08:16:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:12 smithi102 ceph-mon[29849]: osdmap e692: 8 total, 8 up, 8 in 2024-09-11T08:16:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:16:14.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:13 smithi102 ceph-mon[41371]: pgmap v1082: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-11T08:16:14.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:16:14.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:13 smithi102 ceph-mon[41371]: osdmap e693: 8 total, 8 up, 8 in 2024-09-11T08:16:14.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:16:14.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:13 smithi102 ceph-mon[29849]: pgmap v1082: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-11T08:16:14.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:16:14.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:13 smithi102 ceph-mon[29849]: osdmap e693: 8 total, 8 up, 8 in 2024-09-11T08:16:14.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:16:14.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:13 smithi146 ceph-mon[33158]: pgmap v1082: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-11T08:16:14.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:16:14.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:13 smithi146 ceph-mon[33158]: osdmap e693: 8 total, 8 up, 8 in 2024-09-11T08:16:14.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:16:14.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:16:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:16:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:16:15.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:15 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:16:15.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:16:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:15 smithi146 ceph-mon[33158]: pgmap v1085: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:16:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:15 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:16:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:15 smithi146 ceph-mon[33158]: osdmap e694: 8 total, 8 up, 8 in 2024-09-11T08:16:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:15 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:16:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "257.10", "id": [3, 4]}]: dispatch 2024-09-11T08:16:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:15 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "257.10", "id": [3, 4]}]: dispatch 2024-09-11T08:16:15.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:15 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:16:15.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:16:15.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:15 smithi102 ceph-mon[29849]: pgmap v1085: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:16:15.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:15 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:16:15.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:15 smithi102 ceph-mon[29849]: osdmap e694: 8 total, 8 up, 8 in 2024-09-11T08:16:15.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:15 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:16:15.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "257.10", "id": [3, 4]}]: dispatch 2024-09-11T08:16:15.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:15 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "257.10", "id": [3, 4]}]: dispatch 2024-09-11T08:16:15.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:15 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:16:15.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:16:15.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:15 smithi102 ceph-mon[41371]: pgmap v1085: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:16:15.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:15 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:16:15.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:15 smithi102 ceph-mon[41371]: osdmap e694: 8 total, 8 up, 8 in 2024-09-11T08:16:15.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:15 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:16:15.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "257.10", "id": [3, 4]}]: dispatch 2024-09-11T08:16:15.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:15 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "257.10", "id": [3, 4]}]: dispatch 2024-09-11T08:16:16.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:16 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:16:16.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:16:16.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:16 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "257.10", "id": [3, 4]}]': finished 2024-09-11T08:16:16.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:16 smithi146 ceph-mon[33158]: osdmap e695: 8 total, 8 up, 8 in 2024-09-11T08:16:16.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:16:16.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:16 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:16:16.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:16:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:16 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "257.10", "id": [3, 4]}]': finished 2024-09-11T08:16:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:16 smithi102 ceph-mon[41371]: osdmap e695: 8 total, 8 up, 8 in 2024-09-11T08:16:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:16:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:16 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:16:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:16:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:16 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "257.10", "id": [3, 4]}]': finished 2024-09-11T08:16:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:16 smithi102 ceph-mon[29849]: osdmap e695: 8 total, 8 up, 8 in 2024-09-11T08:16:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:16:18.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:17 smithi102 ceph-mon[41371]: pgmap v1088: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:16:18.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:16:18.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:17 smithi102 ceph-mon[41371]: osdmap e696: 8 total, 8 up, 8 in 2024-09-11T08:16:18.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-11T08:16:18.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:17 smithi102 ceph-mon[29849]: pgmap v1088: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:16:18.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:16:18.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:17 smithi102 ceph-mon[29849]: osdmap e696: 8 total, 8 up, 8 in 2024-09-11T08:16:18.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-11T08:16:18.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:17 smithi146 ceph-mon[33158]: pgmap v1088: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:16:18.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:16:18.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:17 smithi146 ceph-mon[33158]: osdmap e696: 8 total, 8 up, 8 in 2024-09-11T08:16:18.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-11T08:16:19.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-11T08:16:19.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:18 smithi102 ceph-mon[41371]: osdmap e697: 8 total, 8 up, 8 in 2024-09-11T08:16:19.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-11T08:16:19.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:18 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-11T08:16:19.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:18 smithi102 ceph-mon[41371]: osdmap e698: 8 total, 8 up, 8 in 2024-09-11T08:16:19.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-11T08:16:19.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:18 smithi102 ceph-mon[29849]: osdmap e697: 8 total, 8 up, 8 in 2024-09-11T08:16:19.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-11T08:16:19.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:18 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-11T08:16:19.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:18 smithi102 ceph-mon[29849]: osdmap e698: 8 total, 8 up, 8 in 2024-09-11T08:16:19.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-11T08:16:19.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:18 smithi146 ceph-mon[33158]: osdmap e697: 8 total, 8 up, 8 in 2024-09-11T08:16:19.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-11T08:16:19.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:18 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-11T08:16:19.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:18 smithi146 ceph-mon[33158]: osdmap e698: 8 total, 8 up, 8 in 2024-09-11T08:16:20.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:19 smithi102 ceph-mon[41371]: pgmap v1091: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:16:20.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:16:20.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:19 smithi102 ceph-mon[29849]: pgmap v1091: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:16:20.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:16:20.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:16:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:16:19] "GET /metrics HTTP/1.1" 200 228711 "" "Prometheus/2.43.0" 2024-09-11T08:16:20.150 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:19 smithi146 ceph-mon[33158]: pgmap v1091: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:16:20.150 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:16:20.433 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:16:20 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:16:21.046 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:16:21.046 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:20 smithi102 ceph-mon[29849]: osdmap e699: 8 total, 8 up, 8 in 2024-09-11T08:16:21.046 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-142"}]: dispatch 2024-09-11T08:16:21.046 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:16:21.047 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:20 smithi102 ceph-mon[41371]: osdmap e699: 8 total, 8 up, 8 in 2024-09-11T08:16:21.047 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-142"}]: dispatch 2024-09-11T08:16:21.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:16:21.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:20 smithi146 ceph-mon[33158]: osdmap e699: 8 total, 8 up, 8 in 2024-09-11T08:16:21.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-142"}]: dispatch 2024-09-11T08:16:22.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:21 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:16:22.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:21 smithi102 ceph-mon[29849]: pgmap v1094: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:16:22.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:21 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-142"}]': finished 2024-09-11T08:16:22.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:21 smithi102 ceph-mon[29849]: osdmap e700: 8 total, 8 up, 8 in 2024-09-11T08:16:22.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:21 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:16:22.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:21 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-142"}]: dispatch 2024-09-11T08:16:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:21 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:16:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:21 smithi102 ceph-mon[41371]: pgmap v1094: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:16:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:21 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-142"}]': finished 2024-09-11T08:16:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:21 smithi102 ceph-mon[41371]: osdmap e700: 8 total, 8 up, 8 in 2024-09-11T08:16:22.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:21 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:16:22.397 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:21 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-142"}]: dispatch 2024-09-11T08:16:22.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:21 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:16:22.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:21 smithi146 ceph-mon[33158]: pgmap v1094: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:16:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:21 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-142"}]': finished 2024-09-11T08:16:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:21 smithi146 ceph-mon[33158]: osdmap e700: 8 total, 8 up, 8 in 2024-09-11T08:16:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:21 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:16:22.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:21 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-142"}]: dispatch 2024-09-11T08:16:23.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:22 smithi102 ceph-mon[29849]: osdmap e701: 8 total, 8 up, 8 in 2024-09-11T08:16:23.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:22 smithi102 ceph-mon[29849]: osdmap e702: 8 total, 8 up, 8 in 2024-09-11T08:16:23.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:16:23.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:22 smithi102 ceph-mon[41371]: osdmap e701: 8 total, 8 up, 8 in 2024-09-11T08:16:23.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:22 smithi102 ceph-mon[41371]: osdmap e702: 8 total, 8 up, 8 in 2024-09-11T08:16:23.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:16:23.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:22 smithi146 ceph-mon[33158]: osdmap e701: 8 total, 8 up, 8 in 2024-09-11T08:16:23.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:22 smithi146 ceph-mon[33158]: osdmap e702: 8 total, 8 up, 8 in 2024-09-11T08:16:23.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:16:24.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:23 smithi102 ceph-mon[29849]: pgmap v1097: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:16:24.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:16:24.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:23 smithi102 ceph-mon[29849]: osdmap e703: 8 total, 8 up, 8 in 2024-09-11T08:16:24.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:16:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:23 smithi102 ceph-mon[41371]: pgmap v1097: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:16:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:16:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:23 smithi102 ceph-mon[41371]: osdmap e703: 8 total, 8 up, 8 in 2024-09-11T08:16:24.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:16:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:23 smithi146 ceph-mon[33158]: pgmap v1097: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:16:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:16:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:23 smithi146 ceph-mon[33158]: osdmap e703: 8 total, 8 up, 8 in 2024-09-11T08:16:24.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:16:24.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:16:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:16:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:16:26.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:25 smithi102 ceph-mon[29849]: pgmap v1100: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:16:26.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:16:26.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:25 smithi102 ceph-mon[29849]: osdmap e704: 8 total, 8 up, 8 in 2024-09-11T08:16:26.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:25 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-144"}]: dispatch 2024-09-11T08:16:26.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:25 smithi102 ceph-mon[41371]: pgmap v1100: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:16:26.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:16:26.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:25 smithi102 ceph-mon[41371]: osdmap e704: 8 total, 8 up, 8 in 2024-09-11T08:16:26.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:25 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-144"}]: dispatch 2024-09-11T08:16:26.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:25 smithi146 ceph-mon[33158]: pgmap v1100: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:16:26.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:16:26.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:25 smithi146 ceph-mon[33158]: osdmap e704: 8 total, 8 up, 8 in 2024-09-11T08:16:26.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:25 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-144"}]: dispatch 2024-09-11T08:16:27.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-144"}]': finished 2024-09-11T08:16:27.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:26 smithi102 ceph-mon[41371]: osdmap e705: 8 total, 8 up, 8 in 2024-09-11T08:16:27.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-144", "mode": "readproxy"}]: dispatch 2024-09-11T08:16:27.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:26 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:16:27.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:26 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-144", "mode": "readproxy"}]': finished 2024-09-11T08:16:27.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:26 smithi102 ceph-mon[41371]: osdmap e706: 8 total, 8 up, 8 in 2024-09-11T08:16:27.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-144"}]': finished 2024-09-11T08:16:27.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:26 smithi102 ceph-mon[29849]: osdmap e705: 8 total, 8 up, 8 in 2024-09-11T08:16:27.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-144", "mode": "readproxy"}]: dispatch 2024-09-11T08:16:27.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:26 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:16:27.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:26 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-144", "mode": "readproxy"}]': finished 2024-09-11T08:16:27.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:26 smithi102 ceph-mon[29849]: osdmap e706: 8 total, 8 up, 8 in 2024-09-11T08:16:27.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-144"}]': finished 2024-09-11T08:16:27.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:26 smithi146 ceph-mon[33158]: osdmap e705: 8 total, 8 up, 8 in 2024-09-11T08:16:27.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-144", "mode": "readproxy"}]: dispatch 2024-09-11T08:16:27.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:26 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:16:27.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:26 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-144", "mode": "readproxy"}]': finished 2024-09-11T08:16:27.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:26 smithi146 ceph-mon[33158]: osdmap e706: 8 total, 8 up, 8 in 2024-09-11T08:16:28.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:27 smithi102 ceph-mon[41371]: pgmap v1103: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:16:28.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:27 smithi102 ceph-mon[29849]: pgmap v1103: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:16:28.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:27 smithi146 ceph-mon[33158]: pgmap v1103: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:16:30.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:16:29 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:16:29] "GET /metrics HTTP/1.1" 200 228751 "" "Prometheus/2.43.0" 2024-09-11T08:16:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:29 smithi102 ceph-mon[41371]: pgmap v1105: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 209 B/s wr, 1 op/s 2024-09-11T08:16:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:29 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:16:30.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:29 smithi102 ceph-mon[29849]: pgmap v1105: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 209 B/s wr, 1 op/s 2024-09-11T08:16:30.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:29 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:16:30.152 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:29 smithi146 ceph-mon[33158]: pgmap v1105: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 209 B/s wr, 1 op/s 2024-09-11T08:16:30.152 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:29 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:16:30.433 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:16:30 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:16:32.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:31 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:16:32.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:31 smithi102 ceph-mon[29849]: pgmap v1106: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 1 op/s 2024-09-11T08:16:32.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:31 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:16:32.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:31 smithi102 ceph-mon[41371]: pgmap v1106: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 1 op/s 2024-09-11T08:16:32.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:31 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:16:32.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:31 smithi146 ceph-mon[33158]: pgmap v1106: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 1 op/s 2024-09-11T08:16:34.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:33 smithi102 ceph-mon[41371]: pgmap v1107: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 129 B/s wr, 1 op/s 2024-09-11T08:16:34.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:33 smithi102 ceph-mon[29849]: pgmap v1107: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 129 B/s wr, 1 op/s 2024-09-11T08:16:34.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:33 smithi146 ceph-mon[33158]: pgmap v1107: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 129 B/s wr, 1 op/s 2024-09-11T08:16:34.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:16:34 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:16:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:16:36.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:35 smithi102 ceph-mon[41371]: pgmap v1108: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 691 B/s rd, 0 op/s 2024-09-11T08:16:36.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:35 smithi102 ceph-mon[29849]: pgmap v1108: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 691 B/s rd, 0 op/s 2024-09-11T08:16:36.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:35 smithi146 ceph-mon[33158]: pgmap v1108: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 691 B/s rd, 0 op/s 2024-09-11T08:16:38.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:37 smithi102 ceph-mon[41371]: pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:16:38.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:16:38.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:37 smithi102 ceph-mon[29849]: pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:16:38.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:16:38.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:37 smithi146 ceph-mon[33158]: pgmap v1109: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:16:38.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:16:39.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:16:39.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:38 smithi102 ceph-mon[41371]: osdmap e707: 8 total, 8 up, 8 in 2024-09-11T08:16:39.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-144"}]: dispatch 2024-09-11T08:16:39.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:16:39.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:38 smithi102 ceph-mon[29849]: osdmap e707: 8 total, 8 up, 8 in 2024-09-11T08:16:39.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-144"}]: dispatch 2024-09-11T08:16:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:16:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:38 smithi146 ceph-mon[33158]: osdmap e707: 8 total, 8 up, 8 in 2024-09-11T08:16:39.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-144"}]: dispatch 2024-09-11T08:16:40.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:16:39 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:16:39] "GET /metrics HTTP/1.1" 200 228751 "" "Prometheus/2.43.0" 2024-09-11T08:16:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:40 smithi102 ceph-mon[41371]: pgmap v1111: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:16:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:40 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:16:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-144"}]': finished 2024-09-11T08:16:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:40 smithi102 ceph-mon[41371]: osdmap e708: 8 total, 8 up, 8 in 2024-09-11T08:16:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:16:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-144"}]: dispatch 2024-09-11T08:16:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:40 smithi102 ceph-mon[29849]: pgmap v1111: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:16:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:40 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:16:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-144"}]': finished 2024-09-11T08:16:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:40 smithi102 ceph-mon[29849]: osdmap e708: 8 total, 8 up, 8 in 2024-09-11T08:16:40.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:16:40.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-144"}]: dispatch 2024-09-11T08:16:40.433 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:16:40 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:16:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:40 smithi146 ceph-mon[33158]: pgmap v1111: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:16:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:40 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:16:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-144"}]': finished 2024-09-11T08:16:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:40 smithi146 ceph-mon[33158]: osdmap e708: 8 total, 8 up, 8 in 2024-09-11T08:16:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:16:40.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-144"}]: dispatch 2024-09-11T08:16:41.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:41 smithi102 ceph-mon[41371]: osdmap e709: 8 total, 8 up, 8 in 2024-09-11T08:16:41.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:41 smithi102 ceph-mon[41371]: osdmap e710: 8 total, 8 up, 8 in 2024-09-11T08:16:41.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:16:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:41 smithi102 ceph-mon[29849]: osdmap e709: 8 total, 8 up, 8 in 2024-09-11T08:16:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:41 smithi102 ceph-mon[29849]: osdmap e710: 8 total, 8 up, 8 in 2024-09-11T08:16:41.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:16:41.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:41 smithi146 ceph-mon[33158]: osdmap e709: 8 total, 8 up, 8 in 2024-09-11T08:16:41.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:41 smithi146 ceph-mon[33158]: osdmap e710: 8 total, 8 up, 8 in 2024-09-11T08:16:41.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:16:42.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:42 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:16:42.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:42 smithi102 ceph-mon[29849]: pgmap v1114: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:16:42.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:16:42.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:42 smithi102 ceph-mon[29849]: osdmap e711: 8 total, 8 up, 8 in 2024-09-11T08:16:42.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:42 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:16:42.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:42 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:16:42.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:42 smithi102 ceph-mon[41371]: pgmap v1114: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:16:42.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:16:42.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:42 smithi102 ceph-mon[41371]: osdmap e711: 8 total, 8 up, 8 in 2024-09-11T08:16:42.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:42 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:16:42.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:42 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:16:42.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:42 smithi146 ceph-mon[33158]: pgmap v1114: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-11T08:16:42.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:16:42.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:42 smithi146 ceph-mon[33158]: osdmap e711: 8 total, 8 up, 8 in 2024-09-11T08:16:42.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:42 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:16:44.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:43 smithi102 ceph-mon[29849]: pgmap v1117: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:16:44.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:43 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:16:44.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:43 smithi102 ceph-mon[29849]: osdmap e712: 8 total, 8 up, 8 in 2024-09-11T08:16:44.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:43 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-146"}]: dispatch 2024-09-11T08:16:44.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:43 smithi102 ceph-mon[41371]: pgmap v1117: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:16:44.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:43 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:16:44.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:43 smithi102 ceph-mon[41371]: osdmap e712: 8 total, 8 up, 8 in 2024-09-11T08:16:44.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:43 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-146"}]: dispatch 2024-09-11T08:16:44.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:43 smithi146 ceph-mon[33158]: pgmap v1117: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:16:44.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:43 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:16:44.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:43 smithi146 ceph-mon[33158]: osdmap e712: 8 total, 8 up, 8 in 2024-09-11T08:16:44.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:43 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-146"}]: dispatch 2024-09-11T08:16:44.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:16:44 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:16:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:16:44.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:44 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-146"}]': finished 2024-09-11T08:16:44.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:44 smithi146 ceph-mon[33158]: osdmap e713: 8 total, 8 up, 8 in 2024-09-11T08:16:44.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:44 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-146", "mode": "writeback"}]: dispatch 2024-09-11T08:16:44.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:44 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:16:44.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:44 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:16:45.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:44 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-146"}]': finished 2024-09-11T08:16:45.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:44 smithi102 ceph-mon[29849]: osdmap e713: 8 total, 8 up, 8 in 2024-09-11T08:16:45.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:44 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-146", "mode": "writeback"}]: dispatch 2024-09-11T08:16:45.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:44 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:16:45.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:44 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:16:45.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:44 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi102-80667-111", "overlaypool": "test-rados-api-smithi102-80667-146"}]': finished 2024-09-11T08:16:45.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:44 smithi102 ceph-mon[41371]: osdmap e713: 8 total, 8 up, 8 in 2024-09-11T08:16:45.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:44 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-146", "mode": "writeback"}]: dispatch 2024-09-11T08:16:45.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:44 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:16:45.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:44 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:16:46.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:45 smithi102 ceph-mon[41371]: pgmap v1120: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:16:46.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:45 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:16:46.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:45 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-146", "mode": "writeback"}]': finished 2024-09-11T08:16:46.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:45 smithi102 ceph-mon[41371]: osdmap e714: 8 total, 8 up, 8 in 2024-09-11T08:16:46.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:45 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:16:46.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:45 smithi102 ceph-mon[29849]: pgmap v1120: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:16:46.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:45 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:16:46.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:45 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-146", "mode": "writeback"}]': finished 2024-09-11T08:16:46.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:45 smithi102 ceph-mon[29849]: osdmap e714: 8 total, 8 up, 8 in 2024-09-11T08:16:46.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:45 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:16:46.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:45 smithi146 ceph-mon[33158]: pgmap v1120: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:16:46.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:45 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-11T08:16:46.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:45 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi102-80667-146", "mode": "writeback"}]': finished 2024-09-11T08:16:46.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:45 smithi146 ceph-mon[33158]: osdmap e714: 8 total, 8 up, 8 in 2024-09-11T08:16:46.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:45 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "hit_set_count","val": "2"}]: dispatch 2024-09-11T08:16:47.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:16:47.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:46 smithi102 ceph-mon[29849]: osdmap e715: 8 total, 8 up, 8 in 2024-09-11T08:16:47.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:16:47.147 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:46 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:16:47.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:46 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:16:47.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:46 smithi102 ceph-mon[41371]: osdmap e715: 8 total, 8 up, 8 in 2024-09-11T08:16:47.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:46 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:16:47.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:46 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:16:47.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:46 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "hit_set_count","val": "2"}]': finished 2024-09-11T08:16:47.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:46 smithi146 ceph-mon[33158]: osdmap e715: 8 total, 8 up, 8 in 2024-09-11T08:16:47.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:46 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "hit_set_period","val": "600"}]: dispatch 2024-09-11T08:16:47.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:46 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "hit_set_period","val": "600"}]': finished 2024-09-11T08:16:48.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:47 smithi102 ceph-mon[29849]: pgmap v1123: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-11T08:16:48.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:47 smithi102 ceph-mon[29849]: osdmap e716: 8 total, 8 up, 8 in 2024-09-11T08:16:48.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:16:48.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:47 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:16:48.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:16:48.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:47 smithi102 ceph-mon[29849]: osdmap e717: 8 total, 8 up, 8 in 2024-09-11T08:16:48.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:47 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:16:48.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:47 smithi102 ceph-mon[41371]: pgmap v1123: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-11T08:16:48.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:47 smithi102 ceph-mon[41371]: osdmap e716: 8 total, 8 up, 8 in 2024-09-11T08:16:48.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:16:48.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:47 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:16:48.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:16:48.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:47 smithi102 ceph-mon[41371]: osdmap e717: 8 total, 8 up, 8 in 2024-09-11T08:16:48.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:47 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:16:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:47 smithi146 ceph-mon[33158]: pgmap v1123: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-11T08:16:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:47 smithi146 ceph-mon[33158]: osdmap e716: 8 total, 8 up, 8 in 2024-09-11T08:16:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-11T08:16:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:47 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-11T08:16:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-11T08:16:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:47 smithi146 ceph-mon[33158]: osdmap e717: 8 total, 8 up, 8 in 2024-09-11T08:16:48.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:47 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-11T08:16:50.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:16:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:16:49] "GET /metrics HTTP/1.1" 200 228682 "" "Prometheus/2.43.0" 2024-09-11T08:16:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:49 smithi102 ceph-mon[29849]: pgmap v1126: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-11T08:16:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:49 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:16:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:49 smithi102 ceph-mon[29849]: osdmap e718: 8 total, 8 up, 8 in 2024-09-11T08:16:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:49 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "target_max_objects","val": "1"}]: dispatch 2024-09-11T08:16:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:49 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:16:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:49 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:16:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:49 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:16:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:49 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:16:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:49 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:16:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:49 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:16:50.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:49 smithi102 ceph-mon[41371]: pgmap v1126: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-11T08:16:50.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:49 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:16:50.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:49 smithi102 ceph-mon[41371]: osdmap e718: 8 total, 8 up, 8 in 2024-09-11T08:16:50.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:49 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "target_max_objects","val": "1"}]: dispatch 2024-09-11T08:16:50.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:49 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:16:50.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:49 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:16:50.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:49 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:16:50.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:49 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:16:50.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:49 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:16:50.148 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:49 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:16:50.169 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:49 smithi146 ceph-mon[33158]: pgmap v1126: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-11T08:16:50.169 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:49 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-11T08:16:50.169 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:49 smithi146 ceph-mon[33158]: osdmap e718: 8 total, 8 up, 8 in 2024-09-11T08:16:50.169 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:49 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "target_max_objects","val": "1"}]: dispatch 2024-09-11T08:16:50.169 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:49 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:16:50.169 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:49 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:16:50.169 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:49 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:16:50.169 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:49 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:16:50.170 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:49 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:16:50.170 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:49 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:16:50.433 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:16:50 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:16:51.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:50 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "target_max_objects","val": "1"}]': finished 2024-09-11T08:16:51.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:50 smithi102 ceph-mon[29849]: osdmap e719: 8 total, 8 up, 8 in 2024-09-11T08:16:51.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:50 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "target_max_objects","val": "1"}]': finished 2024-09-11T08:16:51.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:50 smithi102 ceph-mon[41371]: osdmap e719: 8 total, 8 up, 8 in 2024-09-11T08:16:51.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:50 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-146","var": "target_max_objects","val": "1"}]': finished 2024-09-11T08:16:51.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:50 smithi146 ceph-mon[33158]: osdmap e719: 8 total, 8 up, 8 in 2024-09-11T08:16:52.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:51 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:16:52.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:51 smithi102 ceph-mon[29849]: pgmap v1129: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:16:52.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:51 smithi102 ceph-mon[29849]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-11T08:16:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:51 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:16:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:51 smithi102 ceph-mon[41371]: pgmap v1129: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:16:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:51 smithi102 ceph-mon[41371]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-11T08:16:52.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:51 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:16:52.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:51 smithi146 ceph-mon[33158]: pgmap v1129: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:16:52.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:51 smithi146 ceph-mon[33158]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-11T08:16:54.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:53 smithi102 ceph-mon[41371]: pgmap v1130: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 879 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:16:54.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:53 smithi102 ceph-mon[29849]: pgmap v1130: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 879 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:16:54.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:53 smithi146 ceph-mon[33158]: pgmap v1130: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 879 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:16:54.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:16:54 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:16:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:16:56.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:55 smithi102 ceph-mon[41371]: pgmap v1131: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 750 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:16:56.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:55 smithi102 ceph-mon[29849]: pgmap v1131: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 750 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:16:56.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:55 smithi146 ceph-mon[33158]: pgmap v1131: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 750 B/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:16:58.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:57 smithi102 ceph-mon[41371]: pgmap v1132: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:16:58.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:57 smithi102 ceph-mon[29849]: pgmap v1132: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:16:58.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:57 smithi146 ceph-mon[33158]: pgmap v1132: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:16:59.683 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:16:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a[66648]: logger=sqlstore.transactions t=2024-09-11T08:16:59.400235547Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T08:17:00.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:59 smithi102 ceph-mon[29849]: pgmap v1133: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:17:00.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:59 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:17:00.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:59 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:17:00.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:16:59 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:00.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:16:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:16:59] "GET /metrics HTTP/1.1" 200 228747 "" "Prometheus/2.43.0" 2024-09-11T08:17:00.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:59 smithi102 ceph-mon[41371]: pgmap v1133: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:17:00.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:59 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:17:00.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:59 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:17:00.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:16:59 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:00.180 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:59 smithi146 ceph-mon[33158]: pgmap v1133: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T08:17:00.180 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:59 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:17:00.180 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:59 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:17:00.180 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:16:59 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:00.433 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:17:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:17:01.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:01 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:17:01.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:01 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:17:01.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:01 smithi146 ceph-mon[33158]: osdmap e720: 8 total, 8 up, 8 in 2024-09-11T08:17:01.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:01 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-146"}]: dispatch 2024-09-11T08:17:01.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:01 smithi146 ceph-mon[33158]: pgmap v1135: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:17:01.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:01 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:17:01.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:01 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:17:01.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:01 smithi102 ceph-mon[29849]: osdmap e720: 8 total, 8 up, 8 in 2024-09-11T08:17:01.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:01 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-146"}]: dispatch 2024-09-11T08:17:01.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:01 smithi102 ceph-mon[29849]: pgmap v1135: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:17:01.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:01 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:17:01.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:01 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:17:01.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:01 smithi102 ceph-mon[41371]: osdmap e720: 8 total, 8 up, 8 in 2024-09-11T08:17:01.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:01 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-146"}]: dispatch 2024-09-11T08:17:01.897 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:01 smithi102 ceph-mon[41371]: pgmap v1135: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T08:17:02.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:02 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-146"}]': finished 2024-09-11T08:17:02.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:02 smithi146 ceph-mon[33158]: osdmap e721: 8 total, 8 up, 8 in 2024-09-11T08:17:02.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:02 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:02.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:02 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-146"}]: dispatch 2024-09-11T08:17:02.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:02 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-146"}]': finished 2024-09-11T08:17:02.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:02 smithi102 ceph-mon[29849]: osdmap e721: 8 total, 8 up, 8 in 2024-09-11T08:17:02.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:02 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:02.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:02 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-146"}]: dispatch 2024-09-11T08:17:02.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:02 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-146"}]': finished 2024-09-11T08:17:02.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:02 smithi102 ceph-mon[41371]: osdmap e721: 8 total, 8 up, 8 in 2024-09-11T08:17:02.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:02 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:02.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:02 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-146"}]: dispatch 2024-09-11T08:17:03.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:03 smithi146 ceph-mon[33158]: osdmap e722: 8 total, 8 up, 8 in 2024-09-11T08:17:03.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:03 smithi146 ceph-mon[33158]: pgmap v1138: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-11T08:17:03.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:03 smithi102 ceph-mon[29849]: osdmap e722: 8 total, 8 up, 8 in 2024-09-11T08:17:03.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:03 smithi102 ceph-mon[29849]: pgmap v1138: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-11T08:17:03.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:03 smithi102 ceph-mon[41371]: osdmap e722: 8 total, 8 up, 8 in 2024-09-11T08:17:03.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:03 smithi102 ceph-mon[41371]: pgmap v1138: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-11T08:17:04.683 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:17:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:17:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:17:04.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:04 smithi146 ceph-mon[33158]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-11T08:17:04.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:04 smithi146 ceph-mon[33158]: osdmap e723: 8 total, 8 up, 8 in 2024-09-11T08:17:04.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:04 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:17:04.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:04 smithi102 ceph-mon[41371]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-11T08:17:04.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:04 smithi102 ceph-mon[41371]: osdmap e723: 8 total, 8 up, 8 in 2024-09-11T08:17:04.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:04 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:17:04.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:04 smithi102 ceph-mon[29849]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-11T08:17:04.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:04 smithi102 ceph-mon[29849]: osdmap e723: 8 total, 8 up, 8 in 2024-09-11T08:17:04.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:04 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:17:05.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:05 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:17:05.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:05 smithi146 ceph-mon[33158]: osdmap e724: 8 total, 8 up, 8 in 2024-09-11T08:17:05.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:05 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:17:05.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:05 smithi146 ceph-mon[33158]: pgmap v1141: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:17:05.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:05 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:17:05.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:05 smithi102 ceph-mon[29849]: osdmap e724: 8 total, 8 up, 8 in 2024-09-11T08:17:05.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:05 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:17:05.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:05 smithi102 ceph-mon[29849]: pgmap v1141: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:17:05.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:05 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:17:05.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:05 smithi102 ceph-mon[41371]: osdmap e724: 8 total, 8 up, 8 in 2024-09-11T08:17:05.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:05 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-11T08:17:05.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:05 smithi102 ceph-mon[41371]: pgmap v1141: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:17:06.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:06 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:17:06.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:06 smithi102 ceph-mon[29849]: osdmap e725: 8 total, 8 up, 8 in 2024-09-11T08:17:06.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:06 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-148"}]: dispatch 2024-09-11T08:17:06.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:06 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:17:06.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:06 smithi102 ceph-mon[41371]: osdmap e725: 8 total, 8 up, 8 in 2024-09-11T08:17:06.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:06 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-148"}]: dispatch 2024-09-11T08:17:06.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:06 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-11T08:17:06.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:06 smithi146 ceph-mon[33158]: osdmap e725: 8 total, 8 up, 8 in 2024-09-11T08:17:06.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:06 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-148"}]: dispatch 2024-09-11T08:17:07.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-148"}]': finished 2024-09-11T08:17:07.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:07 smithi102 ceph-mon[29849]: osdmap e726: 8 total, 8 up, 8 in 2024-09-11T08:17:07.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:07.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:07 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-148"}]: dispatch 2024-09-11T08:17:07.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:07 smithi102 ceph-mon[29849]: pgmap v1144: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:17:07.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-148"}]': finished 2024-09-11T08:17:07.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:07 smithi102 ceph-mon[41371]: osdmap e726: 8 total, 8 up, 8 in 2024-09-11T08:17:07.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:07.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:07 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-148"}]: dispatch 2024-09-11T08:17:07.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:07 smithi102 ceph-mon[41371]: pgmap v1144: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:17:07.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-148"}]': finished 2024-09-11T08:17:07.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:07 smithi146 ceph-mon[33158]: osdmap e726: 8 total, 8 up, 8 in 2024-09-11T08:17:07.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:07.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:07 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-148"}]: dispatch 2024-09-11T08:17:07.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:07 smithi146 ceph-mon[33158]: pgmap v1144: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-11T08:17:08.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:08 smithi102 ceph-mon[29849]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:17:08.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:08 smithi102 ceph-mon[29849]: osdmap e727: 8 total, 8 up, 8 in 2024-09-11T08:17:08.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:08 smithi102 ceph-mon[41371]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:17:08.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:08 smithi102 ceph-mon[41371]: osdmap e727: 8 total, 8 up, 8 in 2024-09-11T08:17:08.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:08 smithi146 ceph-mon[33158]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:17:08.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:08 smithi146 ceph-mon[33158]: osdmap e727: 8 total, 8 up, 8 in 2024-09-11T08:17:09.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:09 smithi102 ceph-mon[41371]: osdmap e728: 8 total, 8 up, 8 in 2024-09-11T08:17:09.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:09 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:17:09.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:09 smithi102 ceph-mon[41371]: pgmap v1147: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:17:09.849 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:09 smithi102 ceph-mon[29849]: osdmap e728: 8 total, 8 up, 8 in 2024-09-11T08:17:09.849 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:09 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:17:09.849 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:09 smithi102 ceph-mon[29849]: pgmap v1147: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:17:09.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:09 smithi146 ceph-mon[33158]: osdmap e728: 8 total, 8 up, 8 in 2024-09-11T08:17:09.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:09 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:17:09.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:09 smithi146 ceph-mon[33158]: pgmap v1147: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T08:17:10.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:17:09 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:17:09] "GET /metrics HTTP/1.1" 200 228747 "" "Prometheus/2.43.0" 2024-09-11T08:17:10.683 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:17:10 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:17:11.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:17:11.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:10 smithi102 ceph-mon[29849]: osdmap e729: 8 total, 8 up, 8 in 2024-09-11T08:17:11.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:11.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:10 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-150"}]: dispatch 2024-09-11T08:17:11.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:10 smithi102 ceph-mon[29849]: osdmap e730: 8 total, 8 up, 8 in 2024-09-11T08:17:11.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:17:11.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:10 smithi102 ceph-mon[41371]: osdmap e729: 8 total, 8 up, 8 in 2024-09-11T08:17:11.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:11.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:10 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-150"}]: dispatch 2024-09-11T08:17:11.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:10 smithi102 ceph-mon[41371]: osdmap e730: 8 total, 8 up, 8 in 2024-09-11T08:17:11.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:17:11.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:10 smithi146 ceph-mon[33158]: osdmap e729: 8 total, 8 up, 8 in 2024-09-11T08:17:11.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:11.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:10 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-150"}]: dispatch 2024-09-11T08:17:11.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:10 smithi146 ceph-mon[33158]: osdmap e730: 8 total, 8 up, 8 in 2024-09-11T08:17:12.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:11 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:17:12.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:11 smithi102 ceph-mon[41371]: pgmap v1150: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:17:12.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:11 smithi102 ceph-mon[41371]: osdmap e731: 8 total, 8 up, 8 in 2024-09-11T08:17:12.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:11 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:17:12.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:11 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:17:12.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:11 smithi102 ceph-mon[29849]: pgmap v1150: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:17:12.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:11 smithi102 ceph-mon[29849]: osdmap e731: 8 total, 8 up, 8 in 2024-09-11T08:17:12.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:11 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:17:12.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:11 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:17:12.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:11 smithi146 ceph-mon[33158]: pgmap v1150: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T08:17:12.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:11 smithi146 ceph-mon[33158]: osdmap e731: 8 total, 8 up, 8 in 2024-09-11T08:17:12.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:11 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:17:13.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:17:13.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:13 smithi102 ceph-mon[41371]: osdmap e732: 8 total, 8 up, 8 in 2024-09-11T08:17:13.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:13.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-152"}]: dispatch 2024-09-11T08:17:13.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:13 smithi102 ceph-mon[41371]: pgmap v1153: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:17:13.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:17:13.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:13 smithi102 ceph-mon[29849]: osdmap e732: 8 total, 8 up, 8 in 2024-09-11T08:17:13.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:13.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-152"}]: dispatch 2024-09-11T08:17:13.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:13 smithi102 ceph-mon[29849]: pgmap v1153: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:17:13.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:17:13.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:13 smithi146 ceph-mon[33158]: osdmap e732: 8 total, 8 up, 8 in 2024-09-11T08:17:13.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:13.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-152"}]: dispatch 2024-09-11T08:17:13.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:13 smithi146 ceph-mon[33158]: pgmap v1153: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T08:17:14.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:14 smithi102 ceph-mon[41371]: osdmap e733: 8 total, 8 up, 8 in 2024-09-11T08:17:14.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:14 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:17:14.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:14 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:17:14.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:14 smithi102 ceph-mon[29849]: osdmap e733: 8 total, 8 up, 8 in 2024-09-11T08:17:14.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:14 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:17:14.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:14 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:17:14.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:17:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:17:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:17:14.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:14 smithi146 ceph-mon[33158]: osdmap e733: 8 total, 8 up, 8 in 2024-09-11T08:17:14.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:14 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:17:14.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:14 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:17:15.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:15 smithi102 ceph-mon[41371]: osdmap e734: 8 total, 8 up, 8 in 2024-09-11T08:17:15.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:15 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:17:15.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:15 smithi102 ceph-mon[41371]: pgmap v1156: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:17:15.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:15 smithi102 ceph-mon[29849]: osdmap e734: 8 total, 8 up, 8 in 2024-09-11T08:17:15.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:15 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:17:15.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:15 smithi102 ceph-mon[29849]: pgmap v1156: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:17:15.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:15 smithi146 ceph-mon[33158]: osdmap e734: 8 total, 8 up, 8 in 2024-09-11T08:17:15.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:15 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-11T08:17:15.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:15 smithi146 ceph-mon[33158]: pgmap v1156: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:17:16.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:16 smithi102 ceph-mon[41371]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:17:16.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:17:16.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:16 smithi102 ceph-mon[41371]: osdmap e735: 8 total, 8 up, 8 in 2024-09-11T08:17:16.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-111","var": "dedup_tier","val": "test-rados-api-smithi102-80667-154"}]: dispatch 2024-09-11T08:17:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:16 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-154"}]: dispatch 2024-09-11T08:17:16.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:16 smithi102 ceph-mon[41371]: osdmap e736: 8 total, 8 up, 8 in 2024-09-11T08:17:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:16 smithi102 ceph-mon[29849]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:17:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:17:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:16 smithi102 ceph-mon[29849]: osdmap e735: 8 total, 8 up, 8 in 2024-09-11T08:17:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-111","var": "dedup_tier","val": "test-rados-api-smithi102-80667-154"}]: dispatch 2024-09-11T08:17:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:16 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-154"}]: dispatch 2024-09-11T08:17:16.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:16 smithi102 ceph-mon[29849]: osdmap e736: 8 total, 8 up, 8 in 2024-09-11T08:17:16.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:16 smithi146 ceph-mon[33158]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:17:16.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi102-80667-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-11T08:17:16.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:16 smithi146 ceph-mon[33158]: osdmap e735: 8 total, 8 up, 8 in 2024-09-11T08:17:16.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi102-80667-111","var": "dedup_tier","val": "test-rados-api-smithi102-80667-154"}]: dispatch 2024-09-11T08:17:16.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:16.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:16 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi102-80667-111", "tierpool": "test-rados-api-smithi102-80667-154"}]: dispatch 2024-09-11T08:17:16.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:16 smithi146 ceph-mon[33158]: osdmap e736: 8 total, 8 up, 8 in 2024-09-11T08:17:17.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:17 smithi102 ceph-mon[29849]: pgmap v1159: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-11T08:17:17.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:17 smithi102 ceph-mon[29849]: osdmap e737: 8 total, 8 up, 8 in 2024-09-11T08:17:17.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:17 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:17.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:17 smithi102 ceph-mon[41371]: pgmap v1159: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-11T08:17:17.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:17 smithi102 ceph-mon[41371]: osdmap e737: 8 total, 8 up, 8 in 2024-09-11T08:17:17.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:17 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:17.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:17 smithi146 ceph-mon[33158]: pgmap v1159: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-11T08:17:17.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:17 smithi146 ceph-mon[33158]: osdmap e737: 8 total, 8 up, 8 in 2024-09-11T08:17:17.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:17 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:19.516 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (7890 ms) 2024-09-11T08:17:19.516 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-09-11T08:17:19.516 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (12644 ms) 2024-09-11T08:17:19.516 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-09-11T08:17:19.516 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8054 ms) 2024-09-11T08:17:19.517 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-09-11T08:17:19.517 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (12299 ms) 2024-09-11T08:17:19.517 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-09-11T08:17:19.517 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (8044 ms) 2024-09-11T08:17:19.517 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-09-11T08:17:19.517 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8053 ms) 2024-09-11T08:17:19.517 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-09-11T08:17:19.517 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: hmm, no HitSet yet 2024-09-11T08:17:19.517 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: ok, hit_set contains 255:602f83fe:::foo:head 2024-09-11T08:17:19.517 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9025 ms) 2024-09-11T08:17:19.517 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-09-11T08:17:19.517 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042555,0 2024-09-11T08:17:19.517 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: first is 1726042555 2024-09-11T08:17:19.518 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042555,0 2024-09-11T08:17:19.518 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042555,0 2024-09-11T08:17:19.518 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042555,1726042557,1726042558,0 2024-09-11T08:17:19.518 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042555,1726042557,1726042558,0 2024-09-11T08:17:19.518 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042555,1726042557,1726042558,0 2024-09-11T08:17:19.518 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042555,1726042557,1726042558,1726042560,1726042561,0 2024-09-11T08:17:19.518 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042555,1726042557,1726042558,1726042560,1726042561,0 2024-09-11T08:17:19.518 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042555,1726042557,1726042558,1726042560,1726042561,0 2024-09-11T08:17:19.518 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042555,1726042557,1726042558,1726042560,1726042561,1726042563,1726042564,0 2024-09-11T08:17:19.518 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042555,1726042557,1726042558,1726042560,1726042561,1726042563,1726042564,0 2024-09-11T08:17:19.518 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042555,1726042557,1726042558,1726042560,1726042561,1726042563,1726042564,0 2024-09-11T08:17:19.518 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: got ls 1726042558,1726042560,1726042561,1726042563,1726042564,1726042566,1726042567,0 2024-09-11T08:17:19.519 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: first now 1726042558, trimmed 2024-09-11T08:17:19.519 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (20539 ms) 2024-09-11T08:17:19.519 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-09-11T08:17:19.519 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: foo0 2024-09-11T08:17:19.519 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-09-11T08:17:19.519 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (14050 ms) 2024-09-11T08:17:19.519 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-09-11T08:17:19.519 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (18049 ms) 2024-09-11T08:17:19.519 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-09-11T08:17:19.519 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (22648 ms) 2024-09-11T08:17:19.519 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-09-11T08:17:19.519 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5042 ms) 2024-09-11T08:17:19.519 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-09-11T08:17:19.519 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3019 ms) 2024-09-11T08:17:19.520 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-09-11T08:17:19.520 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3017 ms) 2024-09-11T08:17:19.520 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-09-11T08:17:19.520 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3010 ms) 2024-09-11T08:17:19.520 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (230555 ms total) 2024-09-11T08:17:19.520 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: 2024-09-11T08:17:19.520 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-09-11T08:17:19.520 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (851271 ms total) 2024-09-11T08:17:19.520 INFO:tasks.workunit.client.0.smithi102.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-09-11T08:17:19.524 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.524 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80313 2024-09-11T08:17:19.524 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80313 2024-09-11T08:17:19.525 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.525 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80554 2024-09-11T08:17:19.525 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80554 2024-09-11T08:17:19.525 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.525 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=81097 2024-09-11T08:17:19.525 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 81097 2024-09-11T08:17:19.525 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.525 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80821 2024-09-11T08:17:19.525 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80821 2024-09-11T08:17:19.525 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.525 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80633 2024-09-11T08:17:19.525 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80633 2024-09-11T08:17:19.525 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.526 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80415 2024-09-11T08:17:19.526 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80415 2024-09-11T08:17:19.526 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.526 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80707 2024-09-11T08:17:19.526 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80707 2024-09-11T08:17:19.526 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.526 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80476 2024-09-11T08:17:19.526 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80476 2024-09-11T08:17:19.526 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.526 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80992 2024-09-11T08:17:19.526 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80992 2024-09-11T08:17:19.526 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.526 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80287 2024-09-11T08:17:19.526 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80287 2024-09-11T08:17:19.526 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.527 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80351 2024-09-11T08:17:19.527 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80351 2024-09-11T08:17:19.527 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.527 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=81009 2024-09-11T08:17:19.527 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 81009 2024-09-11T08:17:19.527 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.527 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80278 2024-09-11T08:17:19.527 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80278 2024-09-11T08:17:19.527 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.527 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80298 2024-09-11T08:17:19.527 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80298 2024-09-11T08:17:19.527 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.527 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80566 2024-09-11T08:17:19.527 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80566 2024-09-11T08:17:19.528 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.528 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80672 2024-09-11T08:17:19.528 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80672 2024-09-11T08:17:19.528 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.528 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80599 2024-09-11T08:17:19.528 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80599 2024-09-11T08:17:19.528 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.528 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=81063 2024-09-11T08:17:19.528 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 81063 2024-09-11T08:17:19.528 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.528 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80981 2024-09-11T08:17:19.528 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80981 2024-09-11T08:17:19.528 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.528 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80529 2024-09-11T08:17:19.529 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80529 2024-09-11T08:17:19.529 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.529 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80853 2024-09-11T08:17:19.529 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80853 2024-09-11T08:17:19.529 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.529 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80271 2024-09-11T08:17:19.529 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80271 2024-09-11T08:17:19.529 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.529 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80331 2024-09-11T08:17:19.529 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80331 2024-09-11T08:17:19.529 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.529 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80760 2024-09-11T08:17:19.529 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80760 2024-09-11T08:17:19.529 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.530 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80378 2024-09-11T08:17:19.530 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80378 2024-09-11T08:17:19.530 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.530 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80899 2024-09-11T08:17:19.530 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80899 2024-09-11T08:17:19.530 INFO:tasks.workunit.client.0.smithi102.stderr:+ for t in "${!pids[@]}" 2024-09-11T08:17:19.530 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=80937 2024-09-11T08:17:19.530 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 80937 2024-09-11T08:17:19.530 INFO:tasks.workunit.client.0.smithi102.stderr:+ exit 0 2024-09-11T08:17:19.530 INFO:tasks.workunit.client.0.smithi102.stderr:+ cleanup 2024-09-11T08:17:19.530 INFO:tasks.workunit.client.0.smithi102.stderr:+ pkill -P 80266 2024-09-11T08:17:19.626 INFO:tasks.workunit.client.0.smithi102.stderr:+ true 2024-09-11T08:17:19.626 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T08:17:19.627 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T08:17:19.660 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-09-11T08:17:19.661 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-09-11T08:17:19.661 DEBUG:teuthology.orchestra.run.smithi102:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d0e6828a2016d48cf25ad84064e50742bb1c39b9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-09-11T08:17:19.724 INFO:tasks.workunit.client.0.smithi102.stderr:++ uuidgen 2024-09-11T08:17:19.730 INFO:tasks.workunit.client.0.smithi102.stderr:+ p=91aaf327-ed21-437a-8aea-bd29873f531c 2024-09-11T08:17:19.730 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph osd pool create 91aaf327-ed21-437a-8aea-bd29873f531c 12 2024-09-11T08:17:19.816 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.814+0000 7f47fe19d640 1 -- 172.21.15.102:0/1583522961 >> v1:172.21.15.102:6790/0 conn(0x7f47f8108e90 legacy=0x7f47f810b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:19.817 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.814+0000 7f47fe19d640 1 -- 172.21.15.102:0/1583522961 shutdown_connections 2024-09-11T08:17:19.817 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.814+0000 7f47fe19d640 1 -- 172.21.15.102:0/1583522961 >> 172.21.15.102:0/1583522961 conn(0x7f47f80f8100 msgr2=0x7f47f80fa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:17:19.817 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.814+0000 7f47fe19d640 1 -- 172.21.15.102:0/1583522961 shutdown_connections 2024-09-11T08:17:19.817 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.814+0000 7f47fe19d640 1 -- 172.21.15.102:0/1583522961 wait complete. 2024-09-11T08:17:19.817 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.815+0000 7f47fe19d640 1 Processor -- start 2024-09-11T08:17:19.817 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.815+0000 7f47fe19d640 1 -- start start 2024-09-11T08:17:19.819 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.815+0000 7f47fe19d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f47f811bd40 con 0x7f47f8108e90 2024-09-11T08:17:19.819 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.815+0000 7f47fe19d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f47f811cf00 con 0x7f47f8105300 2024-09-11T08:17:19.819 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.815+0000 7f47fe19d640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f47f811e100 con 0x7f47f8102540 2024-09-11T08:17:19.819 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.815+0000 7f47f77fe640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7f47f8102540 0x7f47f80fe0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:35584/0 (socket says 172.21.15.102:35584) 2024-09-11T08:17:19.819 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.815+0000 7f47f77fe640 1 -- 172.21.15.102:0/3898881565 learned_addr learned my addr 172.21.15.102:0/3898881565 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:17:19.819 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.816+0000 7f47dffff640 1 -- 172.21.15.102:0/3898881565 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1105774234 0 0) 0x7f47f811e100 con 0x7f47f8102540 2024-09-11T08:17:19.820 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.816+0000 7f47dffff640 1 -- 172.21.15.102:0/3898881565 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47e00034e0 con 0x7f47f8102540 2024-09-11T08:17:19.820 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.816+0000 7f47dffff640 1 -- 172.21.15.102:0/3898881565 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 899981445 0 0) 0x7f47f811bd40 con 0x7f47f8108e90 2024-09-11T08:17:19.820 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.816+0000 7f47dffff640 1 -- 172.21.15.102:0/3898881565 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47f811e100 con 0x7f47f8108e90 2024-09-11T08:17:19.820 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.816+0000 7f47dffff640 1 -- 172.21.15.102:0/3898881565 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2449983411 0 0) 0x7f47e00034e0 con 0x7f47f8102540 2024-09-11T08:17:19.820 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.816+0000 7f47dffff640 1 -- 172.21.15.102:0/3898881565 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f47f811bd40 con 0x7f47f8102540 2024-09-11T08:17:19.820 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.816+0000 7f47dffff640 1 -- 172.21.15.102:0/3898881565 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f47e8001970 con 0x7f47f8102540 2024-09-11T08:17:19.820 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.816+0000 7f47dffff640 1 -- 172.21.15.102:0/3898881565 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777612788 0 0) 0x7f47f811cf00 con 0x7f47f8105300 2024-09-11T08:17:19.820 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.816+0000 7f47dffff640 1 -- 172.21.15.102:0/3898881565 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47e00034e0 con 0x7f47f8105300 2024-09-11T08:17:19.820 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.816+0000 7f47dffff640 1 -- 172.21.15.102:0/3898881565 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2040762853 0 0) 0x7f47f811e100 con 0x7f47f8108e90 2024-09-11T08:17:19.820 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.816+0000 7f47dffff640 1 -- 172.21.15.102:0/3898881565 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f47f811cf00 con 0x7f47f8108e90 2024-09-11T08:17:19.820 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.816+0000 7f47dffff640 1 -- 172.21.15.102:0/3898881565 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f47f00016f0 con 0x7f47f8108e90 2024-09-11T08:17:19.820 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.816+0000 7f47dffff640 1 -- 172.21.15.102:0/3898881565 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1256761772 0 0) 0x7f47f811bd40 con 0x7f47f8102540 2024-09-11T08:17:19.820 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.816+0000 7f47dffff640 1 -- 172.21.15.102:0/3898881565 >> v1:172.21.15.146:6789/0 conn(0x7f47f8105300 legacy=0x7f47f80fe800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:19.821 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.816+0000 7f47dffff640 1 -- 172.21.15.102:0/3898881565 >> v1:172.21.15.102:6789/0 conn(0x7f47f8108e90 legacy=0x7f47f811a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:19.821 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.816+0000 7f47dffff640 1 -- 172.21.15.102:0/3898881565 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f811f300 con 0x7f47f8102540 2024-09-11T08:17:19.821 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.816+0000 7f47fe19d640 1 -- 172.21.15.102:0/3898881565 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f811bf30 con 0x7f47f8102540 2024-09-11T08:17:19.821 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.816+0000 7f47fe19d640 1 -- 172.21.15.102:0/3898881565 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f811c400 con 0x7f47f8102540 2024-09-11T08:17:19.821 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.816+0000 7f47dffff640 1 -- 172.21.15.102:0/3898881565 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f47e80023c0 con 0x7f47f8102540 2024-09-11T08:17:19.821 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.816+0000 7f47dffff640 1 -- 172.21.15.102:0/3898881565 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f47e8005a20 con 0x7f47f8102540 2024-09-11T08:17:19.821 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.817+0000 7f47fe19d640 1 -- 172.21.15.102:0/3898881565 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e4005010 con 0x7f47f8102540 2024-09-11T08:17:19.822 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.818+0000 7f47dffff640 1 -- 172.21.15.102:0/3898881565 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7f47e8001f70 con 0x7f47f8102540 2024-09-11T08:17:19.822 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.818+0000 7f47dffff640 1 -- 172.21.15.102:0/3898881565 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(739..739 src has 1..739) v4 ==== 7331+0+0 (unknown 297005649 0 0) 0x7f47e808a480 con 0x7f47f8102540 2024-09-11T08:17:19.824 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.823+0000 7f47dffff640 1 -- 172.21.15.102:0/3898881565 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f47e8056d40 con 0x7f47f8102540 2024-09-11T08:17:19.849 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:17:19.849 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:19 smithi102 ceph-mon[29849]: osdmap e738: 8 total, 8 up, 8 in 2024-09-11T08:17:19.849 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:19 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:19.849 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:19 smithi102 ceph-mon[29849]: pgmap v1162: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:17:19.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:17:19.849 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:19 smithi102 ceph-mon[41371]: osdmap e738: 8 total, 8 up, 8 in 2024-09-11T08:17:19.850 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:19 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:19.850 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:19 smithi102 ceph-mon[41371]: pgmap v1162: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:17:19.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:17:19.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:19 smithi146 ceph-mon[33158]: osdmap e738: 8 total, 8 up, 8 in 2024-09-11T08:17:19.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:19 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80667-111"}]: dispatch 2024-09-11T08:17:19.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:19 smithi146 ceph-mon[33158]: pgmap v1162: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:17:19.983 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:19.982+0000 7f47fe19d640 1 -- 172.21.15.102:0/3898881565 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pg_num": 12} v 0) v1 -- 0x7f47e4005840 con 0x7f47f8102540 2024-09-11T08:17:20.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:17:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:17:19] "GET /metrics HTTP/1.1" 200 226303 "" "Prometheus/2.43.0" 2024-09-11T08:17:20.507 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:17:20 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:17:20.515 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.514+0000 7f47dffff640 1 -- 172.21.15.102:0/3898881565 <== mon.2 v1:172.21.15.102:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pg_num": 12}]=0 pool '91aaf327-ed21-437a-8aea-bd29873f531c' created v740) v1 ==== 176+0+0 (unknown 659259684 0 0) 0x7f47e80566e0 con 0x7f47f8102540 2024-09-11T08:17:20.612 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.610+0000 7f47fe19d640 1 -- 172.21.15.102:0/3898881565 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pg_num": 12} v 0) v1 -- 0x7f47e40052c0 con 0x7f47f8102540 2024-09-11T08:17:20.613 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.611+0000 7f47dffff640 1 -- 172.21.15.102:0/3898881565 <== mon.2 v1:172.21.15.102:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pg_num": 12}]=0 pool '91aaf327-ed21-437a-8aea-bd29873f531c' already exists v740) v1 ==== 183+0+0 (unknown 195339255 0 0) 0x7f47e80568c0 con 0x7f47f8102540 2024-09-11T08:17:20.613 INFO:tasks.workunit.client.0.smithi102.stderr:pool '91aaf327-ed21-437a-8aea-bd29873f531c' already exists 2024-09-11T08:17:20.615 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.613+0000 7f47fe19d640 1 -- 172.21.15.102:0/3898881565 >> v1:172.21.15.102:6800/4281557748 conn(0x7f47e006fda0 legacy=0x7f47e0072250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:20.615 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.613+0000 7f47fe19d640 1 -- 172.21.15.102:0/3898881565 >> v1:172.21.15.102:6790/0 conn(0x7f47f8102540 legacy=0x7f47f80fe0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:20.615 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.613+0000 7f47fe19d640 1 -- 172.21.15.102:0/3898881565 shutdown_connections 2024-09-11T08:17:20.615 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.613+0000 7f47fe19d640 1 -- 172.21.15.102:0/3898881565 >> 172.21.15.102:0/3898881565 conn(0x7f47f80f8100 msgr2=0x7f47f810c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:17:20.615 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.613+0000 7f47fe19d640 1 -- 172.21.15.102:0/3898881565 shutdown_connections 2024-09-11T08:17:20.615 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.613+0000 7f47fe19d640 1 -- 172.21.15.102:0/3898881565 wait complete. 2024-09-11T08:17:20.624 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph osd pool set-quota 91aaf327-ed21-437a-8aea-bd29873f531c max_objects 10 2024-09-11T08:17:20.704 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.699+0000 7f4b4779d640 1 -- 172.21.15.102:0/1474352051 >> v1:172.21.15.102:6790/0 conn(0x7f4b40108f30 legacy=0x7f4b4010b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:20.704 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.699+0000 7f4b4779d640 1 -- 172.21.15.102:0/1474352051 shutdown_connections 2024-09-11T08:17:20.704 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.699+0000 7f4b4779d640 1 -- 172.21.15.102:0/1474352051 >> 172.21.15.102:0/1474352051 conn(0x7f4b40074ac0 msgr2=0x7f4b400f8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:17:20.704 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.699+0000 7f4b4779d640 1 -- 172.21.15.102:0/1474352051 shutdown_connections 2024-09-11T08:17:20.704 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.699+0000 7f4b4779d640 1 -- 172.21.15.102:0/1474352051 wait complete. 2024-09-11T08:17:20.704 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.700+0000 7f4b4779d640 1 Processor -- start 2024-09-11T08:17:20.704 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.700+0000 7f4b4779d640 1 -- start start 2024-09-11T08:17:20.704 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.700+0000 7f4b4779d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4b4011bf10 con 0x7f4b40108f30 2024-09-11T08:17:20.704 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.700+0000 7f4b4779d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4b4011d110 con 0x7f4b401053d0 2024-09-11T08:17:20.704 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.700+0000 7f4b4779d640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4b4011e310 con 0x7f4b40102660 2024-09-11T08:17:20.705 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.700+0000 7f4b45539640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7f4b40102660 0x7f4b400fe600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:35600/0 (socket says 172.21.15.102:35600) 2024-09-11T08:17:20.705 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.700+0000 7f4b45539640 1 -- 172.21.15.102:0/2080288547 learned_addr learned my addr 172.21.15.102:0/2080288547 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:17:20.705 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.700+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2666065622 0 0) 0x7f4b4011bf10 con 0x7f4b40108f30 2024-09-11T08:17:20.705 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.700+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b2c003200 con 0x7f4b40108f30 2024-09-11T08:17:20.705 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.700+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739250639 0 0) 0x7f4b4011e310 con 0x7f4b40102660 2024-09-11T08:17:20.705 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.700+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b4011bf10 con 0x7f4b40102660 2024-09-11T08:17:20.705 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.700+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3781988850 0 0) 0x7f4b4011d110 con 0x7f4b401053d0 2024-09-11T08:17:20.705 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.700+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b4011e310 con 0x7f4b401053d0 2024-09-11T08:17:20.705 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.701+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1977770654 0 0) 0x7f4b4011bf10 con 0x7f4b40102660 2024-09-11T08:17:20.705 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.701+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4b4011d110 con 0x7f4b40102660 2024-09-11T08:17:20.705 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.701+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1003017464 0 0) 0x7f4b2c003200 con 0x7f4b40108f30 2024-09-11T08:17:20.705 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.701+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4b4011bf10 con 0x7f4b40108f30 2024-09-11T08:17:20.706 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.701+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f4b3c001840 con 0x7f4b40102660 2024-09-11T08:17:20.706 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.701+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f4b38001660 con 0x7f4b40108f30 2024-09-11T08:17:20.706 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.701+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1605131359 0 0) 0x7f4b4011e310 con 0x7f4b401053d0 2024-09-11T08:17:20.706 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.701+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4b2c003200 con 0x7f4b401053d0 2024-09-11T08:17:20.706 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.701+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f4b300018d0 con 0x7f4b401053d0 2024-09-11T08:17:20.706 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.701+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 817267215 0 0) 0x7f4b4011d110 con 0x7f4b40102660 2024-09-11T08:17:20.706 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.701+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 >> v1:172.21.15.146:6789/0 conn(0x7f4b401053d0 legacy=0x7f4b400fed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:20.706 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.701+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 >> v1:172.21.15.102:6789/0 conn(0x7f4b40108f30 legacy=0x7f4b4011a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:20.706 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.701+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4011f510 con 0x7f4b40102660 2024-09-11T08:17:20.706 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.701+0000 7f4b4779d640 1 -- 172.21.15.102:0/2080288547 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b4011e500 con 0x7f4b40102660 2024-09-11T08:17:20.706 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.701+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f4b3c0023d0 con 0x7f4b40102660 2024-09-11T08:17:20.706 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.701+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f4b3c0059a0 con 0x7f4b40102660 2024-09-11T08:17:20.706 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.701+0000 7f4b4779d640 1 -- 172.21.15.102:0/2080288547 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b4011e950 con 0x7f4b40102660 2024-09-11T08:17:20.707 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.702+0000 7f4b4779d640 1 -- 172.21.15.102:0/2080288547 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b400fdd70 con 0x7f4b40102660 2024-09-11T08:17:20.707 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.702+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7f4b3c001f80 con 0x7f4b40102660 2024-09-11T08:17:20.707 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.703+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(740..740 src has 1..740) v4 ==== 7706+0+0 (unknown 260610805 0 0) 0x7f4b3c08a490 con 0x7f4b40102660 2024-09-11T08:17:20.709 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.707+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f4b3c056b70 con 0x7f4b40102660 2024-09-11T08:17:20.863 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:20.861+0000 7f4b4779d640 1 -- 172.21.15.102:0/2080288547 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f4b400ff810 con 0x7f4b40102660 2024-09-11T08:17:20.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:17:20.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:20 smithi102 ceph-mon[29849]: osdmap e739: 8 total, 8 up, 8 in 2024-09-11T08:17:20.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:20 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3898881565' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pg_num": 12}]: dispatch 2024-09-11T08:17:20.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:20 smithi102 ceph-mon[29849]: from='client.47066 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pg_num": 12}]: dispatch 2024-09-11T08:17:20.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:17:20.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:20 smithi102 ceph-mon[41371]: osdmap e739: 8 total, 8 up, 8 in 2024-09-11T08:17:20.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:20 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3898881565' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pg_num": 12}]: dispatch 2024-09-11T08:17:20.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:20 smithi102 ceph-mon[41371]: from='client.47066 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pg_num": 12}]: dispatch 2024-09-11T08:17:20.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/4145366198' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi102-80667-111"}]': finished 2024-09-11T08:17:20.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:20 smithi146 ceph-mon[33158]: osdmap e739: 8 total, 8 up, 8 in 2024-09-11T08:17:20.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:20 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3898881565' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pg_num": 12}]: dispatch 2024-09-11T08:17:20.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:20 smithi146 ceph-mon[33158]: from='client.47066 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pg_num": 12}]: dispatch 2024-09-11T08:17:21.529 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:21.527+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 <== mon.2 v1:172.21.15.102:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 91aaf327-ed21-437a-8aea-bd29873f531c v741) v1 ==== 223+0+0 (unknown 227612783 0 0) 0x7f4b3c056510 con 0x7f4b40102660 2024-09-11T08:17:21.628 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:21.626+0000 7f4b4779d640 1 -- 172.21.15.102:0/2080288547 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f4b400646c0 con 0x7f4b40102660 2024-09-11T08:17:21.797 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:21 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:17:21.797 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:21 smithi102 ceph-mon[29849]: from='client.47066 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pg_num": 12}]': finished 2024-09-11T08:17:21.797 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:21 smithi102 ceph-mon[29849]: osdmap e740: 8 total, 8 up, 8 in 2024-09-11T08:17:21.797 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:21 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3898881565' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pg_num": 12}]: dispatch 2024-09-11T08:17:21.797 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:21 smithi102 ceph-mon[29849]: from='client.47066 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pg_num": 12}]: dispatch 2024-09-11T08:17:21.797 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:21 smithi102 ceph-mon[29849]: pgmap v1165: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:17:21.797 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:21 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2080288547' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:17:21.798 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:21 smithi102 ceph-mon[29849]: from='client.47072 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:17:21.798 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:21 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:17:21.798 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:21 smithi102 ceph-mon[41371]: from='client.47066 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pg_num": 12}]': finished 2024-09-11T08:17:21.798 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:21 smithi102 ceph-mon[41371]: osdmap e740: 8 total, 8 up, 8 in 2024-09-11T08:17:21.798 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:21 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3898881565' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pg_num": 12}]: dispatch 2024-09-11T08:17:21.798 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:21 smithi102 ceph-mon[41371]: from='client.47066 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pg_num": 12}]: dispatch 2024-09-11T08:17:21.798 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:21 smithi102 ceph-mon[41371]: pgmap v1165: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:17:21.798 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:21 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2080288547' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:17:21.798 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:21 smithi102 ceph-mon[41371]: from='client.47072 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:17:21.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:21 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:17:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:21 smithi146 ceph-mon[33158]: from='client.47066 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pg_num": 12}]': finished 2024-09-11T08:17:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:21 smithi146 ceph-mon[33158]: osdmap e740: 8 total, 8 up, 8 in 2024-09-11T08:17:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:21 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3898881565' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pg_num": 12}]: dispatch 2024-09-11T08:17:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:21 smithi146 ceph-mon[33158]: from='client.47066 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pg_num": 12}]: dispatch 2024-09-11T08:17:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:21 smithi146 ceph-mon[33158]: pgmap v1165: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:17:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:21 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2080288547' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:17:21.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:21 smithi146 ceph-mon[33158]: from='client.47072 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:17:22.534 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.532+0000 7f4b35ffb640 1 -- 172.21.15.102:0/2080288547 <== mon.2 v1:172.21.15.102:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 91aaf327-ed21-437a-8aea-bd29873f531c v742) v1 ==== 223+0+0 (unknown 3596613631 0 0) 0x7f4b3c0566f0 con 0x7f4b40102660 2024-09-11T08:17:22.534 INFO:tasks.workunit.client.0.smithi102.stderr:set-quota max_objects = 10 for pool 91aaf327-ed21-437a-8aea-bd29873f531c 2024-09-11T08:17:22.537 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.535+0000 7f4b4779d640 1 -- 172.21.15.102:0/2080288547 >> v1:172.21.15.102:6800/4281557748 conn(0x7f4b2c06fe40 legacy=0x7f4b2c0722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:22.537 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.535+0000 7f4b4779d640 1 -- 172.21.15.102:0/2080288547 >> v1:172.21.15.102:6790/0 conn(0x7f4b40102660 legacy=0x7f4b400fe600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:22.537 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.535+0000 7f4b4779d640 1 -- 172.21.15.102:0/2080288547 shutdown_connections 2024-09-11T08:17:22.537 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.535+0000 7f4b4779d640 1 -- 172.21.15.102:0/2080288547 >> 172.21.15.102:0/2080288547 conn(0x7f4b40074ac0 msgr2=0x7f4b400fab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:17:22.537 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.535+0000 7f4b4779d640 1 -- 172.21.15.102:0/2080288547 shutdown_connections 2024-09-11T08:17:22.537 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.535+0000 7f4b4779d640 1 -- 172.21.15.102:0/2080288547 wait complete. 2024-09-11T08:17:22.549 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph osd pool application enable 91aaf327-ed21-437a-8aea-bd29873f531c rados 2024-09-11T08:17:22.632 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.627+0000 7f68e0e57640 1 -- 172.21.15.102:0/2664384022 >> v1:172.21.15.102:6790/0 conn(0x7f68dc108ec0 legacy=0x7f68dc10b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:22.632 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.628+0000 7f68e0e57640 1 -- 172.21.15.102:0/2664384022 shutdown_connections 2024-09-11T08:17:22.632 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.628+0000 7f68e0e57640 1 -- 172.21.15.102:0/2664384022 >> 172.21.15.102:0/2664384022 conn(0x7f68dc074a40 msgr2=0x7f68dc0f8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:17:22.632 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.628+0000 7f68e0e57640 1 -- 172.21.15.102:0/2664384022 shutdown_connections 2024-09-11T08:17:22.632 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.628+0000 7f68e0e57640 1 -- 172.21.15.102:0/2664384022 wait complete. 2024-09-11T08:17:22.632 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.628+0000 7f68e0e57640 1 Processor -- start 2024-09-11T08:17:22.633 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.628+0000 7f68e0e57640 1 -- start start 2024-09-11T08:17:22.633 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.628+0000 7f68e0e57640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f68dc11bdc0 con 0x7f68dc1025f0 2024-09-11T08:17:22.633 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.628+0000 7f68e0e57640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f68dc11cf80 con 0x7f68dc108ec0 2024-09-11T08:17:22.633 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.628+0000 7f68e0e57640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f68dc11e180 con 0x7f68dc105360 2024-09-11T08:17:22.633 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.628+0000 7f68da59c640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f68dc1025f0 0x7f68dc0fe1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:59960/0 (socket says 172.21.15.102:59960) 2024-09-11T08:17:22.633 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.628+0000 7f68da59c640 1 -- 172.21.15.102:0/274432382 learned_addr learned my addr 172.21.15.102:0/274432382 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:17:22.633 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.628+0000 7f68d9d9b640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7f68dc105360 0x7f68dc0fe8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:35622/0 (socket says 172.21.15.102:35622) 2024-09-11T08:17:22.633 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.629+0000 7f68caffd640 1 -- 172.21.15.102:0/274432382 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1385402690 0 0) 0x7f68dc11e180 con 0x7f68dc105360 2024-09-11T08:17:22.633 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.629+0000 7f68caffd640 1 -- 172.21.15.102:0/274432382 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68c40034e0 con 0x7f68dc105360 2024-09-11T08:17:22.633 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.629+0000 7f68caffd640 1 -- 172.21.15.102:0/274432382 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 854025731 0 0) 0x7f68dc11bdc0 con 0x7f68dc1025f0 2024-09-11T08:17:22.633 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.629+0000 7f68caffd640 1 -- 172.21.15.102:0/274432382 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68dc11e180 con 0x7f68dc1025f0 2024-09-11T08:17:22.633 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.629+0000 7f68caffd640 1 -- 172.21.15.102:0/274432382 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3667973013 0 0) 0x7f68c40034e0 con 0x7f68dc105360 2024-09-11T08:17:22.634 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.629+0000 7f68caffd640 1 -- 172.21.15.102:0/274432382 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f68dc11bdc0 con 0x7f68dc105360 2024-09-11T08:17:22.634 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.629+0000 7f68caffd640 1 -- 172.21.15.102:0/274432382 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f68d00019f0 con 0x7f68dc105360 2024-09-11T08:17:22.634 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.629+0000 7f68caffd640 1 -- 172.21.15.102:0/274432382 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 602875706 0 0) 0x7f68dc11e180 con 0x7f68dc1025f0 2024-09-11T08:17:22.634 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.629+0000 7f68caffd640 1 -- 172.21.15.102:0/274432382 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f68c40034e0 con 0x7f68dc1025f0 2024-09-11T08:17:22.634 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.629+0000 7f68caffd640 1 -- 172.21.15.102:0/274432382 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f68cc002850 con 0x7f68dc1025f0 2024-09-11T08:17:22.634 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.629+0000 7f68caffd640 1 -- 172.21.15.102:0/274432382 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060731298 0 0) 0x7f68dc11cf80 con 0x7f68dc108ec0 2024-09-11T08:17:22.634 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.629+0000 7f68caffd640 1 -- 172.21.15.102:0/274432382 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68dc11e180 con 0x7f68dc108ec0 2024-09-11T08:17:22.634 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.629+0000 7f68caffd640 1 -- 172.21.15.102:0/274432382 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1433034441 0 0) 0x7f68dc11bdc0 con 0x7f68dc105360 2024-09-11T08:17:22.634 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.629+0000 7f68caffd640 1 -- 172.21.15.102:0/274432382 >> v1:172.21.15.146:6789/0 conn(0x7f68dc108ec0 legacy=0x7f68dc11a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:22.634 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.629+0000 7f68caffd640 1 -- 172.21.15.102:0/274432382 >> v1:172.21.15.102:6789/0 conn(0x7f68dc1025f0 legacy=0x7f68dc0fe1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:22.634 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.629+0000 7f68caffd640 1 -- 172.21.15.102:0/274432382 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68dc11f380 con 0x7f68dc105360 2024-09-11T08:17:22.635 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.629+0000 7f68e0e57640 1 -- 172.21.15.102:0/274432382 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68dc11bfb0 con 0x7f68dc105360 2024-09-11T08:17:22.635 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.629+0000 7f68e0e57640 1 -- 172.21.15.102:0/274432382 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f68dc11c480 con 0x7f68dc105360 2024-09-11T08:17:22.635 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.629+0000 7f68caffd640 1 -- 172.21.15.102:0/274432382 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f68d0001c60 con 0x7f68dc105360 2024-09-11T08:17:22.635 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.629+0000 7f68caffd640 1 -- 172.21.15.102:0/274432382 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f68d00059f0 con 0x7f68dc105360 2024-09-11T08:17:22.635 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.630+0000 7f68e0e57640 1 -- 172.21.15.102:0/274432382 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68bc005010 con 0x7f68dc105360 2024-09-11T08:17:22.635 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.630+0000 7f68caffd640 1 -- 172.21.15.102:0/274432382 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7f68d00022b0 con 0x7f68dc105360 2024-09-11T08:17:22.636 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.631+0000 7f68caffd640 1 -- 172.21.15.102:0/274432382 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(742..742 src has 1..742) v4 ==== 7706+0+0 (unknown 2635603405 0 0) 0x7f68d008a4a0 con 0x7f68dc105360 2024-09-11T08:17:22.636 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.635+0000 7f68caffd640 1 -- 172.21.15.102:0/274432382 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f68d0056be0 con 0x7f68dc105360 2024-09-11T08:17:22.793 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:22.791+0000 7f68e0e57640 1 -- 172.21.15.102:0/274432382 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "app": "rados"} v 0) v1 -- 0x7f68bc0052c0 con 0x7f68dc105360 2024-09-11T08:17:22.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:22 smithi102 ceph-mon[41371]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:17:22.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:22 smithi102 ceph-mon[41371]: from='client.47072 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:17:22.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:22 smithi102 ceph-mon[41371]: osdmap e741: 8 total, 8 up, 8 in 2024-09-11T08:17:22.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:22 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2080288547' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:17:22.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:22 smithi102 ceph-mon[41371]: from='client.47072 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:17:22.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:22 smithi102 ceph-mon[29849]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:17:22.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:22 smithi102 ceph-mon[29849]: from='client.47072 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:17:22.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:22 smithi102 ceph-mon[29849]: osdmap e741: 8 total, 8 up, 8 in 2024-09-11T08:17:22.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:22 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2080288547' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:17:22.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:22 smithi102 ceph-mon[29849]: from='client.47072 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:17:22.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:22 smithi146 ceph-mon[33158]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:17:22.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:22 smithi146 ceph-mon[33158]: from='client.47072 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:17:22.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:22 smithi146 ceph-mon[33158]: osdmap e741: 8 total, 8 up, 8 in 2024-09-11T08:17:22.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:22 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2080288547' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:17:22.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:22 smithi146 ceph-mon[33158]: from='client.47072 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:17:23.537 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:23.535+0000 7f68caffd640 1 -- 172.21.15.102:0/274432382 <== mon.2 v1:172.21.15.102:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "app": "rados"}]=0 enabled application 'rados' on pool '91aaf327-ed21-437a-8aea-bd29873f531c' v743) v1 ==== 213+0+0 (unknown 830928204 0 0) 0x7f68d0056580 con 0x7f68dc105360 2024-09-11T08:17:23.631 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:23.629+0000 7f68e0e57640 1 -- 172.21.15.102:0/274432382 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "app": "rados"} v 0) v1 -- 0x7f68bc001090 con 0x7f68dc105360 2024-09-11T08:17:23.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:23 smithi102 ceph-mon[41371]: from='client.47072 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:17:23.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:23 smithi102 ceph-mon[41371]: osdmap e742: 8 total, 8 up, 8 in 2024-09-11T08:17:23.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:23 smithi102 ceph-mon[41371]: pgmap v1168: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:17:23.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:23 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/274432382' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "app": "rados"}]: dispatch 2024-09-11T08:17:23.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:23 smithi102 ceph-mon[41371]: from='client.47078 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "app": "rados"}]: dispatch 2024-09-11T08:17:23.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:23 smithi102 ceph-mon[29849]: from='client.47072 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:17:23.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:23 smithi102 ceph-mon[29849]: osdmap e742: 8 total, 8 up, 8 in 2024-09-11T08:17:23.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:23 smithi102 ceph-mon[29849]: pgmap v1168: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:17:23.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:23 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/274432382' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "app": "rados"}]: dispatch 2024-09-11T08:17:23.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:23 smithi102 ceph-mon[29849]: from='client.47078 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "app": "rados"}]: dispatch 2024-09-11T08:17:23.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:23 smithi146 ceph-mon[33158]: from='client.47072 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:17:23.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:23 smithi146 ceph-mon[33158]: osdmap e742: 8 total, 8 up, 8 in 2024-09-11T08:17:23.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:23 smithi146 ceph-mon[33158]: pgmap v1168: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:17:23.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:23 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/274432382' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "app": "rados"}]: dispatch 2024-09-11T08:17:23.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:23 smithi146 ceph-mon[33158]: from='client.47078 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "app": "rados"}]: dispatch 2024-09-11T08:17:24.538 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:24.537+0000 7f68caffd640 1 -- 172.21.15.102:0/274432382 <== mon.2 v1:172.21.15.102:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "app": "rados"}]=0 enabled application 'rados' on pool '91aaf327-ed21-437a-8aea-bd29873f531c' v744) v1 ==== 213+0+0 (unknown 1132524607 0 0) 0x7f68d0056760 con 0x7f68dc105360 2024-09-11T08:17:24.539 INFO:tasks.workunit.client.0.smithi102.stderr:enabled application 'rados' on pool '91aaf327-ed21-437a-8aea-bd29873f531c' 2024-09-11T08:17:24.541 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:24.539+0000 7f68e0e57640 1 -- 172.21.15.102:0/274432382 >> v1:172.21.15.102:6800/4281557748 conn(0x7f68c406fd30 legacy=0x7f68c40721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:24.541 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:24.539+0000 7f68e0e57640 1 -- 172.21.15.102:0/274432382 >> v1:172.21.15.102:6790/0 conn(0x7f68dc105360 legacy=0x7f68dc0fe8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:24.541 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:24.539+0000 7f68e0e57640 1 -- 172.21.15.102:0/274432382 shutdown_connections 2024-09-11T08:17:24.541 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:24.539+0000 7f68e0e57640 1 -- 172.21.15.102:0/274432382 >> 172.21.15.102:0/274432382 conn(0x7f68dc074a40 msgr2=0x7f68dc0f8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:17:24.541 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:24.539+0000 7f68e0e57640 1 -- 172.21.15.102:0/274432382 shutdown_connections 2024-09-11T08:17:24.541 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:24.539+0000 7f68e0e57640 1 -- 172.21.15.102:0/274432382 wait complete. 2024-09-11T08:17:24.552 INFO:tasks.workunit.client.0.smithi102.stderr:++ seq 1 10 2024-09-11T08:17:24.558 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 10` 2024-09-11T08:17:24.559 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 91aaf327-ed21-437a-8aea-bd29873f531c put obj1 /etc/passwd 2024-09-11T08:17:24.590 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 10` 2024-09-11T08:17:24.590 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 91aaf327-ed21-437a-8aea-bd29873f531c put obj2 /etc/passwd 2024-09-11T08:17:24.623 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 10` 2024-09-11T08:17:24.623 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 91aaf327-ed21-437a-8aea-bd29873f531c put obj3 /etc/passwd 2024-09-11T08:17:24.652 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 10` 2024-09-11T08:17:24.652 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 91aaf327-ed21-437a-8aea-bd29873f531c put obj4 /etc/passwd 2024-09-11T08:17:24.680 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 10` 2024-09-11T08:17:24.681 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 91aaf327-ed21-437a-8aea-bd29873f531c put obj5 /etc/passwd 2024-09-11T08:17:24.709 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 10` 2024-09-11T08:17:24.709 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 91aaf327-ed21-437a-8aea-bd29873f531c put obj6 /etc/passwd 2024-09-11T08:17:24.737 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 10` 2024-09-11T08:17:24.737 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 91aaf327-ed21-437a-8aea-bd29873f531c put obj7 /etc/passwd 2024-09-11T08:17:24.765 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 10` 2024-09-11T08:17:24.765 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 91aaf327-ed21-437a-8aea-bd29873f531c put obj8 /etc/passwd 2024-09-11T08:17:24.794 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 10` 2024-09-11T08:17:24.794 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 91aaf327-ed21-437a-8aea-bd29873f531c put obj9 /etc/passwd 2024-09-11T08:17:24.823 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 10` 2024-09-11T08:17:24.823 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 91aaf327-ed21-437a-8aea-bd29873f531c put obj10 /etc/passwd 2024-09-11T08:17:24.852 INFO:tasks.workunit.client.0.smithi102.stderr:+ sleep 30 2024-09-11T08:17:24.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:24 smithi102 ceph-mon[41371]: from='client.47078 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "app": "rados"}]': finished 2024-09-11T08:17:24.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:24 smithi102 ceph-mon[41371]: osdmap e743: 8 total, 8 up, 8 in 2024-09-11T08:17:24.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:24 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/274432382' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "app": "rados"}]: dispatch 2024-09-11T08:17:24.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:24 smithi102 ceph-mon[41371]: from='client.47078 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "app": "rados"}]: dispatch 2024-09-11T08:17:24.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:24 smithi102 ceph-mon[29849]: from='client.47078 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "app": "rados"}]': finished 2024-09-11T08:17:24.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:24 smithi102 ceph-mon[29849]: osdmap e743: 8 total, 8 up, 8 in 2024-09-11T08:17:24.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:24 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/274432382' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "app": "rados"}]: dispatch 2024-09-11T08:17:24.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:24 smithi102 ceph-mon[29849]: from='client.47078 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "app": "rados"}]: dispatch 2024-09-11T08:17:24.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:24 smithi146 ceph-mon[33158]: from='client.47078 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "app": "rados"}]': finished 2024-09-11T08:17:24.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:24 smithi146 ceph-mon[33158]: osdmap e743: 8 total, 8 up, 8 in 2024-09-11T08:17:24.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:24 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/274432382' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "app": "rados"}]: dispatch 2024-09-11T08:17:24.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:24 smithi146 ceph-mon[33158]: from='client.47078 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "app": "rados"}]: dispatch 2024-09-11T08:17:24.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:17:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:17:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:17:25.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:25 smithi102 ceph-mon[41371]: from='client.47078 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "app": "rados"}]': finished 2024-09-11T08:17:25.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:25 smithi102 ceph-mon[41371]: osdmap e744: 8 total, 8 up, 8 in 2024-09-11T08:17:25.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:25 smithi102 ceph-mon[41371]: pgmap v1171: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:17:25.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:25 smithi102 ceph-mon[29849]: from='client.47078 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "app": "rados"}]': finished 2024-09-11T08:17:25.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:25 smithi102 ceph-mon[29849]: osdmap e744: 8 total, 8 up, 8 in 2024-09-11T08:17:25.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:25 smithi102 ceph-mon[29849]: pgmap v1171: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:17:25.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:25 smithi146 ceph-mon[33158]: from='client.47078 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "app": "rados"}]': finished 2024-09-11T08:17:25.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:25 smithi146 ceph-mon[33158]: osdmap e744: 8 total, 8 up, 8 in 2024-09-11T08:17:25.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:25 smithi146 ceph-mon[33158]: pgmap v1171: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:17:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:27 smithi146 ceph-mon[33158]: pgmap v1172: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1004 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-11T08:17:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:27 smithi146 ceph-mon[33158]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:17:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:27 smithi146 ceph-mon[33158]: pool '91aaf327-ed21-437a-8aea-bd29873f531c' is full (reached quota's max_objects: 10) 2024-09-11T08:17:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:27 smithi146 ceph-mon[33158]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:17:27.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:27 smithi146 ceph-mon[33158]: osdmap e745: 8 total, 8 up, 8 in 2024-09-11T08:17:28.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:27 smithi102 ceph-mon[41371]: pgmap v1172: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1004 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-11T08:17:28.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:27 smithi102 ceph-mon[41371]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:17:28.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:27 smithi102 ceph-mon[41371]: pool '91aaf327-ed21-437a-8aea-bd29873f531c' is full (reached quota's max_objects: 10) 2024-09-11T08:17:28.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:27 smithi102 ceph-mon[41371]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:17:28.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:27 smithi102 ceph-mon[41371]: osdmap e745: 8 total, 8 up, 8 in 2024-09-11T08:17:28.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:27 smithi102 ceph-mon[29849]: pgmap v1172: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1004 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-11T08:17:28.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:27 smithi102 ceph-mon[29849]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T08:17:28.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:27 smithi102 ceph-mon[29849]: pool '91aaf327-ed21-437a-8aea-bd29873f531c' is full (reached quota's max_objects: 10) 2024-09-11T08:17:28.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:27 smithi102 ceph-mon[29849]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:17:28.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:27 smithi102 ceph-mon[29849]: osdmap e745: 8 total, 8 up, 8 in 2024-09-11T08:17:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:29 smithi146 ceph-mon[33158]: pgmap v1174: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T08:17:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:29 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:17:29.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:29 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:17:30.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:17:29 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:17:29] "GET /metrics HTTP/1.1" 200 226308 "" "Prometheus/2.43.0" 2024-09-11T08:17:30.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:29 smithi102 ceph-mon[29849]: pgmap v1174: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T08:17:30.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:29 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:17:30.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:29 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:17:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:29 smithi102 ceph-mon[41371]: pgmap v1174: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T08:17:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:29 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:17:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:29 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:17:30.683 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:17:30 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:17:31.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:31 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:17:31.934 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:31 smithi146 ceph-mon[33158]: pgmap v1175: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 18 KiB/s wr, 2 op/s 2024-09-11T08:17:32.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:31 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:17:32.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:31 smithi102 ceph-mon[29849]: pgmap v1175: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 18 KiB/s wr, 2 op/s 2024-09-11T08:17:32.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:31 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:17:32.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:31 smithi102 ceph-mon[41371]: pgmap v1175: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 18 KiB/s wr, 2 op/s 2024-09-11T08:17:33.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:33 smithi102 ceph-mon[41371]: pgmap v1176: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 16 KiB/s wr, 2 op/s 2024-09-11T08:17:33.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:33 smithi102 ceph-mon[29849]: pgmap v1176: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 16 KiB/s wr, 2 op/s 2024-09-11T08:17:33.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:33 smithi146 ceph-mon[33158]: pgmap v1176: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 16 KiB/s wr, 2 op/s 2024-09-11T08:17:34.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:17:34 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:17:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:17:35.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:35 smithi146 ceph-mon[33158]: pgmap v1177: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:17:36.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:35 smithi102 ceph-mon[41371]: pgmap v1177: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:17:36.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:35 smithi102 ceph-mon[29849]: pgmap v1177: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:17:37.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:37 smithi146 ceph-mon[33158]: pgmap v1178: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:17:38.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:37 smithi102 ceph-mon[29849]: pgmap v1178: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:17:38.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:37 smithi102 ceph-mon[41371]: pgmap v1178: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:17:39.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:39 smithi146 ceph-mon[33158]: pgmap v1179: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 883 B/s rd, 0 op/s 2024-09-11T08:17:40.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:39 smithi102 ceph-mon[41371]: pgmap v1179: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 883 B/s rd, 0 op/s 2024-09-11T08:17:40.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:39 smithi102 ceph-mon[29849]: pgmap v1179: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 883 B/s rd, 0 op/s 2024-09-11T08:17:40.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:17:39 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:17:39] "GET /metrics HTTP/1.1" 200 226308 "" "Prometheus/2.43.0" 2024-09-11T08:17:40.683 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:17:40 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:17:42.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:41 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:17:42.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:41 smithi102 ceph-mon[29849]: pgmap v1180: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:17:42.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:41 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:17:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:41 smithi102 ceph-mon[41371]: pgmap v1180: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:17:42.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:41 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:17:42.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:41 smithi146 ceph-mon[33158]: pgmap v1180: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:17:44.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:43 smithi102 ceph-mon[29849]: pgmap v1181: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:17:44.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:43 smithi102 ceph-mon[41371]: pgmap v1181: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:17:44.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:43 smithi146 ceph-mon[33158]: pgmap v1181: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:17:44.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:44 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:17:44.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:17:44 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:17:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:17:45.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:44 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:17:45.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:44 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:17:46.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:45 smithi102 ceph-mon[41371]: pgmap v1182: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:17:46.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:45 smithi102 ceph-mon[29849]: pgmap v1182: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:17:46.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:45 smithi146 ceph-mon[33158]: pgmap v1182: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:17:48.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:47 smithi102 ceph-mon[41371]: pgmap v1183: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:17:48.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:47 smithi102 ceph-mon[29849]: pgmap v1183: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:17:48.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:47 smithi146 ceph-mon[33158]: pgmap v1183: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:17:50.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:17:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:17:49] "GET /metrics HTTP/1.1" 200 226238 "" "Prometheus/2.43.0" 2024-09-11T08:17:50.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:49 smithi102 ceph-mon[41371]: pgmap v1184: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:17:50.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:49 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:17:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:49 smithi102 ceph-mon[29849]: pgmap v1184: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:17:50.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:49 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:17:50.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:49 smithi146 ceph-mon[33158]: pgmap v1184: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:17:50.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:49 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:17:50.683 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:17:50 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:17:51.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:50 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:17:51.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:50 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:17:51.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:50 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:17:51.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:50 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:17:51.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:50 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:17:51.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:50 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:17:51.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:50 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:17:51.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:50 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:17:51.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:50 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:17:51.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:50 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:17:51.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:50 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:17:51.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:50 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:17:51.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:50 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:17:51.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:50 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:17:51.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:50 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:17:52.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:51 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:17:52.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:51 smithi102 ceph-mon[41371]: pgmap v1185: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:17:52.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:51 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:17:52.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:51 smithi102 ceph-mon[29849]: pgmap v1185: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:17:52.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:51 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:17:52.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:51 smithi146 ceph-mon[33158]: pgmap v1185: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:17:54.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:53 smithi102 ceph-mon[29849]: pgmap v1186: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:17:54.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:53 smithi102 ceph-mon[41371]: pgmap v1186: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:17:54.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:53 smithi146 ceph-mon[33158]: pgmap v1186: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:17:54.854 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=149497 2024-09-11T08:17:54.854 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph osd pool set-quota 91aaf327-ed21-437a-8aea-bd29873f531c max_objects 100 2024-09-11T08:17:54.854 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 91aaf327-ed21-437a-8aea-bd29873f531c put onemore /etc/passwd 2024-09-11T08:17:54.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:17:54 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:17:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:17:54.943 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.942+0000 7efe5715d640 1 -- 172.21.15.102:0/2680090535 >> v1:172.21.15.102:6789/0 conn(0x7efe501052f0 legacy=0x7efe501076d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:54.943 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.942+0000 7efe5715d640 1 -- 172.21.15.102:0/2680090535 shutdown_connections 2024-09-11T08:17:54.943 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.942+0000 7efe5715d640 1 -- 172.21.15.102:0/2680090535 >> 172.21.15.102:0/2680090535 conn(0x7efe500f8130 msgr2=0x7efe500fa540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:17:54.944 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.942+0000 7efe5715d640 1 -- 172.21.15.102:0/2680090535 shutdown_connections 2024-09-11T08:17:54.944 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.942+0000 7efe5715d640 1 -- 172.21.15.102:0/2680090535 wait complete. 2024-09-11T08:17:54.944 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.943+0000 7efe5715d640 1 Processor -- start 2024-09-11T08:17:54.944 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.943+0000 7efe5715d640 1 -- start start 2024-09-11T08:17:54.946 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.943+0000 7efe5715d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7efe5011dfe0 con 0x7efe501052f0 2024-09-11T08:17:54.947 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.943+0000 7efe5715d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7efe5011f180 con 0x7efe50102530 2024-09-11T08:17:54.947 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.943+0000 7efe5715d640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7efe50120320 con 0x7efe50108e80 2024-09-11T08:17:54.947 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.943+0000 7efe4ffff640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7efe501052f0 0x7efe500700b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:60448/0 (socket says 172.21.15.102:60448) 2024-09-11T08:17:54.947 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.943+0000 7efe4ffff640 1 -- 172.21.15.102:0/1405486718 learned_addr learned my addr 172.21.15.102:0/1405486718 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:17:54.947 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.943+0000 7efe556fa640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7efe50108e80 0x7efe5011c8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:50078/0 (socket says 172.21.15.102:50078) 2024-09-11T08:17:54.947 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.944+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3813582019 0 0) 0x7efe5011dfe0 con 0x7efe501052f0 2024-09-11T08:17:54.947 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.944+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe3c0034e0 con 0x7efe501052f0 2024-09-11T08:17:54.947 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.944+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3032165966 0 0) 0x7efe5011f180 con 0x7efe50102530 2024-09-11T08:17:54.947 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.944+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe5011dfe0 con 0x7efe50102530 2024-09-11T08:17:54.947 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.944+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021807006 0 0) 0x7efe50120320 con 0x7efe50108e80 2024-09-11T08:17:54.947 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.944+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe5011f180 con 0x7efe50108e80 2024-09-11T08:17:54.947 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.944+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2006133324 0 0) 0x7efe3c0034e0 con 0x7efe501052f0 2024-09-11T08:17:54.948 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.944+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe50120320 con 0x7efe501052f0 2024-09-11T08:17:54.948 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.944+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7efe38001370 con 0x7efe501052f0 2024-09-11T08:17:54.948 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.944+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1876690673 0 0) 0x7efe5011f180 con 0x7efe50108e80 2024-09-11T08:17:54.948 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.944+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe3c0034e0 con 0x7efe50108e80 2024-09-11T08:17:54.948 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.944+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1265737805 0 0) 0x7efe5011dfe0 con 0x7efe50102530 2024-09-11T08:17:54.948 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.944+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe5011f180 con 0x7efe50102530 2024-09-11T08:17:54.948 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.944+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7efe48001b40 con 0x7efe50108e80 2024-09-11T08:17:54.948 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.944+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7efe40001550 con 0x7efe50102530 2024-09-11T08:17:54.948 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.944+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1219356098 0 0) 0x7efe50120320 con 0x7efe501052f0 2024-09-11T08:17:54.948 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.944+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 >> v1:172.21.15.102:6790/0 conn(0x7efe50108e80 legacy=0x7efe5011c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:54.948 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.944+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 >> v1:172.21.15.146:6789/0 conn(0x7efe50102530 legacy=0x7efe5006f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:54.948 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.944+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe50121500 con 0x7efe501052f0 2024-09-11T08:17:54.949 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.944+0000 7efe5715d640 1 -- 172.21.15.102:0/1405486718 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe5011f370 con 0x7efe501052f0 2024-09-11T08:17:54.949 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.944+0000 7efe5715d640 1 -- 172.21.15.102:0/1405486718 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7efe5011f840 con 0x7efe501052f0 2024-09-11T08:17:54.949 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.944+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7efe380015d0 con 0x7efe501052f0 2024-09-11T08:17:54.949 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.944+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7efe38003680 con 0x7efe501052f0 2024-09-11T08:17:54.949 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.945+0000 7efe5715d640 1 -- 172.21.15.102:0/1405486718 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe30005010 con 0x7efe501052f0 2024-09-11T08:17:54.949 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.946+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7efe380017c0 con 0x7efe501052f0 2024-09-11T08:17:54.949 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.946+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(745..745 src has 1..745) v4 ==== 7719+0+0 (unknown 212395974 0 0) 0x7efe3808a6a0 con 0x7efe501052f0 2024-09-11T08:17:54.950 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.946+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=746}) v3 -- 0x7efe50120320 con 0x7efe501052f0 2024-09-11T08:17:54.951 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:54.951+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7efe38056de0 con 0x7efe501052f0 2024-09-11T08:17:55.110 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:55.109+0000 7efe5715d640 1 -- 172.21.15.102:0/1405486718 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7efe300052c0 con 0x7efe501052f0 2024-09-11T08:17:55.868 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:55.867+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 <== mon.0 v1:172.21.15.102:6789/0 10 ==== osd_map(746..746 src has 1..746) v4 ==== 620+0+0 (unknown 3825688453 0 0) 0x7efe38056b20 con 0x7efe501052f0 2024-09-11T08:17:55.868 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:55.867+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=747}) v3 -- 0x7efe5011dfe0 con 0x7efe501052f0 2024-09-11T08:17:55.871 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:55.871+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 <== mon.0 v1:172.21.15.102:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 91aaf327-ed21-437a-8aea-bd29873f531c v746) v1 ==== 225+0+0 (unknown 2239879883 0 0) 0x7efe38056780 con 0x7efe501052f0 2024-09-11T08:17:55.969 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:55.969+0000 7efe5715d640 1 -- 172.21.15.102:0/1405486718 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7efe30005ad0 con 0x7efe501052f0 2024-09-11T08:17:56.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:55 smithi102 ceph-mon[41371]: pgmap v1187: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:17:56.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:55 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1405486718' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "100"}]: dispatch 2024-09-11T08:17:56.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:55 smithi102 ceph-mon[29849]: pgmap v1187: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:17:56.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:55 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1405486718' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "100"}]: dispatch 2024-09-11T08:17:56.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:55 smithi146 ceph-mon[33158]: pgmap v1187: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:17:56.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:55 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1405486718' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "100"}]: dispatch 2024-09-11T08:17:56.883 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:56.882+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 <== mon.0 v1:172.21.15.102:6789/0 12 ==== osd_map(747..747 src has 1..747) v4 ==== 620+0+0 (unknown 106740788 0 0) 0x7efe38057710 con 0x7efe501052f0 2024-09-11T08:17:56.883 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:56.882+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=748}) v3 -- 0x7efe3c07cb80 con 0x7efe501052f0 2024-09-11T08:17:56.885 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:56.884+0000 7efe4d7fa640 1 -- 172.21.15.102:0/1405486718 <== mon.0 v1:172.21.15.102:6789/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 91aaf327-ed21-437a-8aea-bd29873f531c v747) v1 ==== 225+0+0 (unknown 2908939955 0 0) 0x7efe38056780 con 0x7efe501052f0 2024-09-11T08:17:56.885 INFO:tasks.workunit.client.0.smithi102.stderr:set-quota max_objects = 100 for pool 91aaf327-ed21-437a-8aea-bd29873f531c 2024-09-11T08:17:56.891 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:56.891+0000 7efe5715d640 1 -- 172.21.15.102:0/1405486718 >> v1:172.21.15.102:6800/4281557748 conn(0x7efe3c06ff20 legacy=0x7efe3c0723d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:56.891 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:56.891+0000 7efe5715d640 1 -- 172.21.15.102:0/1405486718 >> v1:172.21.15.102:6789/0 conn(0x7efe501052f0 legacy=0x7efe500700b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:56.892 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:56.891+0000 7efe5715d640 1 -- 172.21.15.102:0/1405486718 shutdown_connections 2024-09-11T08:17:56.892 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:56.891+0000 7efe5715d640 1 -- 172.21.15.102:0/1405486718 >> 172.21.15.102:0/1405486718 conn(0x7efe500f8130 msgr2=0x7efe5010c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:17:56.892 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:56.891+0000 7efe5715d640 1 -- 172.21.15.102:0/1405486718 shutdown_connections 2024-09-11T08:17:56.892 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:56.891+0000 7efe5715d640 1 -- 172.21.15.102:0/1405486718 wait complete. 2024-09-11T08:17:56.903 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 149497 2024-09-11T08:17:57.042 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 0 -ne 0 ']' 2024-09-11T08:17:57.042 INFO:tasks.workunit.client.0.smithi102.stderr:+ true 2024-09-11T08:17:57.042 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 91aaf327-ed21-437a-8aea-bd29873f531c put twomore /etc/passwd 2024-09-11T08:17:57.073 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph osd pool set-quota 91aaf327-ed21-437a-8aea-bd29873f531c max_bytes 100 2024-09-11T08:17:57.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:56 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1405486718' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "100"}]': finished 2024-09-11T08:17:57.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:56 smithi102 ceph-mon[41371]: osdmap e746: 8 total, 8 up, 8 in 2024-09-11T08:17:57.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:56 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1405486718' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "100"}]: dispatch 2024-09-11T08:17:57.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:56 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1405486718' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "100"}]': finished 2024-09-11T08:17:57.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:56 smithi102 ceph-mon[29849]: osdmap e746: 8 total, 8 up, 8 in 2024-09-11T08:17:57.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:56 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1405486718' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "100"}]: dispatch 2024-09-11T08:17:57.154 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.153+0000 7f39b64ef640 1 -- 172.21.15.102:0/3227101827 >> v1:172.21.15.102:6789/0 conn(0x7f39b0107d70 legacy=0x7f39b010a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:57.154 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.153+0000 7f39b64ef640 1 -- 172.21.15.102:0/3227101827 shutdown_connections 2024-09-11T08:17:57.154 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.153+0000 7f39b64ef640 1 -- 172.21.15.102:0/3227101827 >> 172.21.15.102:0/3227101827 conn(0x7f39b00752f0 msgr2=0x7f39b00f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:17:57.154 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.153+0000 7f39b64ef640 1 -- 172.21.15.102:0/3227101827 shutdown_connections 2024-09-11T08:17:57.154 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.153+0000 7f39b64ef640 1 -- 172.21.15.102:0/3227101827 wait complete. 2024-09-11T08:17:57.154 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.153+0000 7f39b64ef640 1 Processor -- start 2024-09-11T08:17:57.157 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.154+0000 7f39b64ef640 1 -- start start 2024-09-11T08:17:57.157 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.154+0000 7f39b64ef640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f39b011beb0 con 0x7f39b0107d70 2024-09-11T08:17:57.157 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.154+0000 7f39b64ef640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f39b011d0b0 con 0x7f39b0104210 2024-09-11T08:17:57.157 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.154+0000 7f39b64ef640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f39b011e2b0 con 0x7f39b01014a0 2024-09-11T08:17:57.157 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.154+0000 7f39affff640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7f39b01014a0 0x7f39b0064860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:50116/0 (socket says 172.21.15.102:50116) 2024-09-11T08:17:57.157 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.154+0000 7f39affff640 1 -- 172.21.15.102:0/3654314519 learned_addr learned my addr 172.21.15.102:0/3654314519 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:17:57.158 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.154+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006024982 0 0) 0x7f39b011e2b0 con 0x7f39b01014a0 2024-09-11T08:17:57.158 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.154+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f399c0034e0 con 0x7f39b01014a0 2024-09-11T08:17:57.158 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.154+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585546448 0 0) 0x7f39b011beb0 con 0x7f39b0107d70 2024-09-11T08:17:57.158 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.154+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39b011e2b0 con 0x7f39b0107d70 2024-09-11T08:17:57.158 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.154+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4289732601 0 0) 0x7f39b011d0b0 con 0x7f39b0104210 2024-09-11T08:17:57.158 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.154+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39b011beb0 con 0x7f39b0104210 2024-09-11T08:17:57.158 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.154+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 335655966 0 0) 0x7f39b011e2b0 con 0x7f39b0107d70 2024-09-11T08:17:57.158 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.154+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f39b011d0b0 con 0x7f39b0107d70 2024-09-11T08:17:57.158 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.154+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3471904689 0 0) 0x7f399c0034e0 con 0x7f39b01014a0 2024-09-11T08:17:57.158 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.154+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f39b011e2b0 con 0x7f39b01014a0 2024-09-11T08:17:57.158 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.154+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f39a80023e0 con 0x7f39b0107d70 2024-09-11T08:17:57.158 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.154+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f39a0001a50 con 0x7f39b01014a0 2024-09-11T08:17:57.158 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.154+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 548694446 0 0) 0x7f39b011beb0 con 0x7f39b0104210 2024-09-11T08:17:57.159 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.154+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f399c0034e0 con 0x7f39b0104210 2024-09-11T08:17:57.159 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.155+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f39980018c0 con 0x7f39b0104210 2024-09-11T08:17:57.159 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.155+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1130797367 0 0) 0x7f39b011d0b0 con 0x7f39b0107d70 2024-09-11T08:17:57.159 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.155+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 >> v1:172.21.15.102:6790/0 conn(0x7f39b01014a0 legacy=0x7f39b0064860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:57.159 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.155+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 >> v1:172.21.15.146:6789/0 conn(0x7f39b0104210 legacy=0x7f39b0064f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:57.159 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.155+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b011f4b0 con 0x7f39b0107d70 2024-09-11T08:17:57.159 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.155+0000 7f39b64ef640 1 -- 172.21.15.102:0/3654314519 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b011c0a0 con 0x7f39b0107d70 2024-09-11T08:17:57.159 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.155+0000 7f39b64ef640 1 -- 172.21.15.102:0/3654314519 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b011c570 con 0x7f39b0107d70 2024-09-11T08:17:57.159 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.155+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f39a8001580 con 0x7f39b0107d70 2024-09-11T08:17:57.159 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.155+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f39a8003700 con 0x7f39b0107d70 2024-09-11T08:17:57.159 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.156+0000 7f39b64ef640 1 -- 172.21.15.102:0/3654314519 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3990005010 con 0x7f39b0107d70 2024-09-11T08:17:57.159 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.156+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7f39a8019c00 con 0x7f39b0107d70 2024-09-11T08:17:57.160 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.157+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(748..748 src has 1..748) v4 ==== 7719+0+0 (unknown 1357323479 0 0) 0x7f39a8088020 con 0x7f39b0107d70 2024-09-11T08:17:57.161 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.161+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f39a8054760 con 0x7f39b0107d70 2024-09-11T08:17:57.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:56 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1405486718' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "100"}]': finished 2024-09-11T08:17:57.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:56 smithi146 ceph-mon[33158]: osdmap e746: 8 total, 8 up, 8 in 2024-09-11T08:17:57.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:56 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1405486718' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "100"}]: dispatch 2024-09-11T08:17:57.317 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:57.316+0000 7f39b64ef640 1 -- 172.21.15.102:0/3654314519 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f39900052c0 con 0x7f39b0107d70 2024-09-11T08:17:58.039 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:58.038+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 91aaf327-ed21-437a-8aea-bd29873f531c v749) v1 ==== 221+0+0 (unknown 3442807771 0 0) 0x7f39a8054100 con 0x7f39b0107d70 2024-09-11T08:17:58.135 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:58.134+0000 7f39b64ef640 1 -- 172.21.15.102:0/3654314519 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f3990000be0 con 0x7f39b0107d70 2024-09-11T08:17:58.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:57 smithi102 ceph-mon[29849]: pgmap v1189: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:17:58.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:57 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1405486718' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "100"}]': finished 2024-09-11T08:17:58.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:57 smithi102 ceph-mon[29849]: osdmap e747: 8 total, 8 up, 8 in 2024-09-11T08:17:58.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:57 smithi102 ceph-mon[29849]: pool '91aaf327-ed21-437a-8aea-bd29873f531c' no longer out of quota; removing NO_QUOTA flag 2024-09-11T08:17:58.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:57 smithi102 ceph-mon[29849]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:17:58.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:57 smithi102 ceph-mon[29849]: osdmap e748: 8 total, 8 up, 8 in 2024-09-11T08:17:58.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:57 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3654314519' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-11T08:17:58.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:57 smithi102 ceph-mon[41371]: pgmap v1189: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:17:58.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:57 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1405486718' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "100"}]': finished 2024-09-11T08:17:58.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:57 smithi102 ceph-mon[41371]: osdmap e747: 8 total, 8 up, 8 in 2024-09-11T08:17:58.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:57 smithi102 ceph-mon[41371]: pool '91aaf327-ed21-437a-8aea-bd29873f531c' no longer out of quota; removing NO_QUOTA flag 2024-09-11T08:17:58.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:57 smithi102 ceph-mon[41371]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:17:58.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:57 smithi102 ceph-mon[41371]: osdmap e748: 8 total, 8 up, 8 in 2024-09-11T08:17:58.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:57 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3654314519' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-11T08:17:58.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:57 smithi146 ceph-mon[33158]: pgmap v1189: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:17:58.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:57 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1405486718' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "100"}]': finished 2024-09-11T08:17:58.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:57 smithi146 ceph-mon[33158]: osdmap e747: 8 total, 8 up, 8 in 2024-09-11T08:17:58.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:57 smithi146 ceph-mon[33158]: pool '91aaf327-ed21-437a-8aea-bd29873f531c' no longer out of quota; removing NO_QUOTA flag 2024-09-11T08:17:58.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:57 smithi146 ceph-mon[33158]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:17:58.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:57 smithi146 ceph-mon[33158]: osdmap e748: 8 total, 8 up, 8 in 2024-09-11T08:17:58.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:57 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3654314519' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-11T08:17:59.043 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:59.042+0000 7f39acff9640 1 -- 172.21.15.102:0/3654314519 <== mon.0 v1:172.21.15.102:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 91aaf327-ed21-437a-8aea-bd29873f531c v750) v1 ==== 221+0+0 (unknown 1210586889 0 0) 0x7f39a80542e0 con 0x7f39b0107d70 2024-09-11T08:17:59.044 INFO:tasks.workunit.client.0.smithi102.stderr:set-quota max_bytes = 100 for pool 91aaf327-ed21-437a-8aea-bd29873f531c 2024-09-11T08:17:59.046 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:59.045+0000 7f39b64ef640 1 -- 172.21.15.102:0/3654314519 >> v1:172.21.15.102:6800/4281557748 conn(0x7f399c06fe10 legacy=0x7f399c0722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:59.046 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:59.045+0000 7f39b64ef640 1 -- 172.21.15.102:0/3654314519 >> v1:172.21.15.102:6789/0 conn(0x7f39b0107d70 legacy=0x7f39b011a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:17:59.046 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:59.045+0000 7f39b64ef640 1 -- 172.21.15.102:0/3654314519 shutdown_connections 2024-09-11T08:17:59.046 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:59.045+0000 7f39b64ef640 1 -- 172.21.15.102:0/3654314519 >> 172.21.15.102:0/3654314519 conn(0x7f39b00752f0 msgr2=0x7f39b010ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:17:59.046 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:59.046+0000 7f39b64ef640 1 -- 172.21.15.102:0/3654314519 shutdown_connections 2024-09-11T08:17:59.046 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:17:59.046+0000 7f39b64ef640 1 -- 172.21.15.102:0/3654314519 wait complete. 2024-09-11T08:17:59.056 INFO:tasks.workunit.client.0.smithi102.stderr:+ sleep 30 2024-09-11T08:17:59.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:59 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3654314519' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "100"}]': finished 2024-09-11T08:17:59.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:59 smithi102 ceph-mon[29849]: osdmap e749: 8 total, 8 up, 8 in 2024-09-11T08:17:59.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:59 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3654314519' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-11T08:17:59.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:17:59 smithi102 ceph-mon[29849]: pgmap v1193: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:17:59.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:59 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3654314519' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "100"}]': finished 2024-09-11T08:17:59.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:59 smithi102 ceph-mon[41371]: osdmap e749: 8 total, 8 up, 8 in 2024-09-11T08:17:59.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:59 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3654314519' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-11T08:17:59.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:17:59 smithi102 ceph-mon[41371]: pgmap v1193: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:17:59.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:59 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3654314519' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "100"}]': finished 2024-09-11T08:17:59.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:59 smithi146 ceph-mon[33158]: osdmap e749: 8 total, 8 up, 8 in 2024-09-11T08:17:59.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:59 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3654314519' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-11T08:17:59.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:17:59 smithi146 ceph-mon[33158]: pgmap v1193: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:18:00.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:17:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:17:59] "GET /metrics HTTP/1.1" 200 226194 "" "Prometheus/2.43.0" 2024-09-11T08:18:00.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:00 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3654314519' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "100"}]': finished 2024-09-11T08:18:00.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:00 smithi102 ceph-mon[29849]: osdmap e750: 8 total, 8 up, 8 in 2024-09-11T08:18:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:00 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:18:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:00 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:18:00.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:00 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3654314519' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "100"}]': finished 2024-09-11T08:18:00.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:00 smithi102 ceph-mon[41371]: osdmap e750: 8 total, 8 up, 8 in 2024-09-11T08:18:00.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:00 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:18:00.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:00 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:18:00.433 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:18:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:18:00.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:00 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3654314519' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "100"}]': finished 2024-09-11T08:18:00.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:00 smithi146 ceph-mon[33158]: osdmap e750: 8 total, 8 up, 8 in 2024-09-11T08:18:00.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:00 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:18:00.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:00 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:18:01.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:01 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:18:01.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:01 smithi102 ceph-mon[29849]: pgmap v1195: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-11T08:18:01.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:01 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:18:01.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:01 smithi102 ceph-mon[41371]: pgmap v1195: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-11T08:18:01.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:01 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:18:01.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:01 smithi146 ceph-mon[33158]: pgmap v1195: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-11T08:18:02.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:02 smithi102 ceph-mon[29849]: pool '91aaf327-ed21-437a-8aea-bd29873f531c' is full (reached quota's max_bytes: 100 B) 2024-09-11T08:18:02.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:02 smithi102 ceph-mon[29849]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:18:02.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:02 smithi102 ceph-mon[29849]: osdmap e751: 8 total, 8 up, 8 in 2024-09-11T08:18:02.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:02 smithi102 ceph-mon[41371]: pool '91aaf327-ed21-437a-8aea-bd29873f531c' is full (reached quota's max_bytes: 100 B) 2024-09-11T08:18:02.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:02 smithi102 ceph-mon[41371]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:18:02.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:02 smithi102 ceph-mon[41371]: osdmap e751: 8 total, 8 up, 8 in 2024-09-11T08:18:02.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:02 smithi146 ceph-mon[33158]: pool '91aaf327-ed21-437a-8aea-bd29873f531c' is full (reached quota's max_bytes: 100 B) 2024-09-11T08:18:02.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:02 smithi146 ceph-mon[33158]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:18:02.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:02 smithi146 ceph-mon[33158]: osdmap e751: 8 total, 8 up, 8 in 2024-09-11T08:18:03.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:03 smithi102 ceph-mon[29849]: pgmap v1197: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 914 B/s rd, 4.6 KiB/s wr, 1 op/s 2024-09-11T08:18:03.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:03 smithi102 ceph-mon[41371]: pgmap v1197: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 914 B/s rd, 4.6 KiB/s wr, 1 op/s 2024-09-11T08:18:03.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:03 smithi146 ceph-mon[33158]: pgmap v1197: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 914 B/s rd, 4.6 KiB/s wr, 1 op/s 2024-09-11T08:18:04.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:18:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:18:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:18:06.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:05 smithi102 ceph-mon[29849]: pgmap v1198: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 775 B/s rd, 3.9 KiB/s wr, 1 op/s 2024-09-11T08:18:06.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:05 smithi102 ceph-mon[41371]: pgmap v1198: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 775 B/s rd, 3.9 KiB/s wr, 1 op/s 2024-09-11T08:18:06.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:05 smithi146 ceph-mon[33158]: pgmap v1198: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 775 B/s rd, 3.9 KiB/s wr, 1 op/s 2024-09-11T08:18:08.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:07 smithi102 ceph-mon[29849]: pgmap v1199: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-11T08:18:08.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:07 smithi102 ceph-mon[41371]: pgmap v1199: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-11T08:18:08.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:07 smithi146 ceph-mon[33158]: pgmap v1199: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-11T08:18:10.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:09 smithi102 ceph-mon[29849]: pgmap v1200: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.7 KiB/s wr, 1 op/s 2024-09-11T08:18:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:09 smithi102 ceph-mon[41371]: pgmap v1200: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.7 KiB/s wr, 1 op/s 2024-09-11T08:18:10.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:18:09 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:18:09] "GET /metrics HTTP/1.1" 200 226194 "" "Prometheus/2.43.0" 2024-09-11T08:18:10.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:09 smithi146 ceph-mon[33158]: pgmap v1200: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.7 KiB/s wr, 1 op/s 2024-09-11T08:18:10.683 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:18:10 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:18:12.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:11 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:18:12.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:11 smithi102 ceph-mon[41371]: pgmap v1201: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:18:12.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:11 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:18:12.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:11 smithi102 ceph-mon[29849]: pgmap v1201: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:18:12.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:11 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:18:12.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:11 smithi146 ceph-mon[33158]: pgmap v1201: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:18:14.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:13 smithi102 ceph-mon[41371]: pgmap v1202: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 965 B/s rd, 0 op/s 2024-09-11T08:18:14.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:13 smithi102 ceph-mon[29849]: pgmap v1202: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 965 B/s rd, 0 op/s 2024-09-11T08:18:14.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:13 smithi146 ceph-mon[33158]: pgmap v1202: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 965 B/s rd, 0 op/s 2024-09-11T08:18:14.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:18:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:18:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:18:15.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:15 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:18:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:18:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:15 smithi146 ceph-mon[33158]: pgmap v1203: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:18:15.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:15 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:18:15.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:18:15.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:15 smithi102 ceph-mon[41371]: pgmap v1203: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:18:15.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:15 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:18:15.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:18:15.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:15 smithi102 ceph-mon[29849]: pgmap v1203: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:18:18.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:17 smithi102 ceph-mon[29849]: pgmap v1204: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:18:18.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:17 smithi102 ceph-mon[41371]: pgmap v1204: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:18:18.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:17 smithi146 ceph-mon[33158]: pgmap v1204: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:18:20.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:19 smithi102 ceph-mon[41371]: pgmap v1205: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:18:20.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:19 smithi102 ceph-mon[29849]: pgmap v1205: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:18:20.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:18:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:18:19] "GET /metrics HTTP/1.1" 200 226240 "" "Prometheus/2.43.0" 2024-09-11T08:18:20.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:19 smithi146 ceph-mon[33158]: pgmap v1205: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:18:20.683 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:18:20 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:18:21.948 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:21 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:18:21.948 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:21 smithi102 ceph-mon[29849]: pgmap v1206: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:18:21.948 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:21 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:18:21.948 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:21 smithi102 ceph-mon[41371]: pgmap v1206: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:18:22.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:21 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:18:22.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:21 smithi146 ceph-mon[33158]: pgmap v1206: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:18:24.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:23 smithi102 ceph-mon[29849]: pgmap v1207: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:18:24.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:23 smithi102 ceph-mon[41371]: pgmap v1207: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:18:24.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:23 smithi146 ceph-mon[33158]: pgmap v1207: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:18:24.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:18:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:18:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:18:26.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:25 smithi102 ceph-mon[29849]: pgmap v1208: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:18:26.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:25 smithi102 ceph-mon[41371]: pgmap v1208: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:18:26.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:25 smithi146 ceph-mon[33158]: pgmap v1208: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:18:28.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:27 smithi102 ceph-mon[29849]: pgmap v1209: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:18:28.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:27 smithi102 ceph-mon[41371]: pgmap v1209: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:18:28.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:27 smithi146 ceph-mon[33158]: pgmap v1209: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:18:29.058 INFO:tasks.workunit.client.0.smithi102.stderr:+ pid=149586 2024-09-11T08:18:29.058 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph osd pool set-quota 91aaf327-ed21-437a-8aea-bd29873f531c max_bytes 0 2024-09-11T08:18:29.059 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 91aaf327-ed21-437a-8aea-bd29873f531c put two /etc/passwd 2024-09-11T08:18:29.144 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.143+0000 7f31a5280640 1 -- 172.21.15.102:0/4029029304 >> v1:172.21.15.102:6789/0 conn(0x7f31a00fd540 legacy=0x7f31a00fd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:29.145 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.143+0000 7f31a5280640 1 -- 172.21.15.102:0/4029029304 shutdown_connections 2024-09-11T08:18:29.145 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.143+0000 7f31a5280640 1 -- 172.21.15.102:0/4029029304 >> 172.21.15.102:0/4029029304 conn(0x7f31a00752f0 msgr2=0x7f31a00f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:18:29.145 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.143+0000 7f31a5280640 1 -- 172.21.15.102:0/4029029304 shutdown_connections 2024-09-11T08:18:29.145 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.143+0000 7f31a5280640 1 -- 172.21.15.102:0/4029029304 wait complete. 2024-09-11T08:18:29.145 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.144+0000 7f31a5280640 1 Processor -- start 2024-09-11T08:18:29.145 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.144+0000 7f31a5280640 1 -- start start 2024-09-11T08:18:29.145 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.144+0000 7f31a5280640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f31a011be90 con 0x7f31a00fe110 2024-09-11T08:18:29.145 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.144+0000 7f31a5280640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f31a011d090 con 0x7f31a0108e70 2024-09-11T08:18:29.145 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.144+0000 7f31a5280640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f31a011e290 con 0x7f31a00fd540 2024-09-11T08:18:29.145 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.144+0000 7f319effd640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7f31a00fd540 0x7f31a01040d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:41042/0 (socket says 172.21.15.102:41042) 2024-09-11T08:18:29.145 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.144+0000 7f319effd640 1 -- 172.21.15.102:0/103230928 learned_addr learned my addr 172.21.15.102:0/103230928 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:18:29.145 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.144+0000 7f319e7fc640 1 --1- 172.21.15.102:0/103230928 >> v1:172.21.15.102:6789/0 conn(0x7f31a00fe110 0x7f31a01047e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:55542/0 (socket says 172.21.15.102:55542) 2024-09-11T08:18:29.148 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.144+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4047666242 0 0) 0x7f31a011e290 con 0x7f31a00fd540 2024-09-11T08:18:29.148 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.144+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f318c003120 con 0x7f31a00fd540 2024-09-11T08:18:29.148 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.144+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1742971424 0 0) 0x7f31a011be90 con 0x7f31a00fe110 2024-09-11T08:18:29.148 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.144+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31a011e290 con 0x7f31a00fe110 2024-09-11T08:18:29.148 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.144+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4175714573 0 0) 0x7f31a011d090 con 0x7f31a0108e70 2024-09-11T08:18:29.148 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.144+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31a011be90 con 0x7f31a0108e70 2024-09-11T08:18:29.149 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.145+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3735260141 0 0) 0x7f318c003120 con 0x7f31a00fd540 2024-09-11T08:18:29.149 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.145+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f31a011d090 con 0x7f31a00fd540 2024-09-11T08:18:29.149 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.145+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3932676949 0 0) 0x7f31a011e290 con 0x7f31a00fe110 2024-09-11T08:18:29.149 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.145+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f318c003120 con 0x7f31a00fe110 2024-09-11T08:18:29.149 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.145+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f3190001650 con 0x7f31a00fd540 2024-09-11T08:18:29.149 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.145+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f3188001a80 con 0x7f31a00fe110 2024-09-11T08:18:29.149 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.145+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 837105101 0 0) 0x7f31a011be90 con 0x7f31a0108e70 2024-09-11T08:18:29.149 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.145+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f31a011e290 con 0x7f31a0108e70 2024-09-11T08:18:29.149 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.145+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f3198001880 con 0x7f31a0108e70 2024-09-11T08:18:29.149 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.145+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1033363364 0 0) 0x7f31a011d090 con 0x7f31a00fd540 2024-09-11T08:18:29.150 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.145+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 >> v1:172.21.15.146:6789/0 conn(0x7f31a0108e70 legacy=0x7f31a011a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:29.150 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.145+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 >> v1:172.21.15.102:6789/0 conn(0x7f31a00fe110 legacy=0x7f31a01047e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:29.150 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.145+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a011f490 con 0x7f31a00fd540 2024-09-11T08:18:29.150 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.145+0000 7f31a5280640 1 -- 172.21.15.102:0/103230928 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a011e480 con 0x7f31a00fd540 2024-09-11T08:18:29.150 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.145+0000 7f31a5280640 1 -- 172.21.15.102:0/103230928 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a011e8d0 con 0x7f31a00fd540 2024-09-11T08:18:29.150 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.145+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f31900014d0 con 0x7f31a00fd540 2024-09-11T08:18:29.150 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.145+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f3190003920 con 0x7f31a00fd540 2024-09-11T08:18:29.150 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.146+0000 7f31a5280640 1 -- 172.21.15.102:0/103230928 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3180005010 con 0x7f31a00fd540 2024-09-11T08:18:29.150 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.147+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7f3190003b20 con 0x7f31a00fd540 2024-09-11T08:18:29.150 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.147+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(751..751 src has 1..751) v4 ==== 7719+0+0 (unknown 2558105719 0 0) 0x7f319008aae0 con 0x7f31a00fd540 2024-09-11T08:18:29.150 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.147+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=752}) v3 -- 0x7f31a011d090 con 0x7f31a00fd540 2024-09-11T08:18:29.151 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.151+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f3190057220 con 0x7f31a00fd540 2024-09-11T08:18:29.310 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.310+0000 7f31a5280640 1 -- 172.21.15.102:0/103230928 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f31800052c0 con 0x7f31a00fd540 2024-09-11T08:18:29.713 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.712+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 <== mon.2 v1:172.21.15.102:6790/0 10 ==== osd_map(752..752 src has 1..752) v4 ==== 620+0+0 (unknown 3162606660 0 0) 0x7f3190056f60 con 0x7f31a00fd540 2024-09-11T08:18:29.713 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.712+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=753}) v3 -- 0x7f318c003120 con 0x7f31a00fd540 2024-09-11T08:18:29.718 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.718+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 <== mon.2 v1:172.21.15.102:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 91aaf327-ed21-437a-8aea-bd29873f531c v752) v1 ==== 217+0+0 (unknown 1102079779 0 0) 0x7f3190056bc0 con 0x7f31a00fd540 2024-09-11T08:18:29.814 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:29.814+0000 7f31a5280640 1 -- 172.21.15.102:0/103230928 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f3180000c40 con 0x7f31a00fd540 2024-09-11T08:18:30.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:18:29 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:18:29] "GET /metrics HTTP/1.1" 200 226235 "" "Prometheus/2.43.0" 2024-09-11T08:18:30.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:29 smithi102 ceph-mon[29849]: pgmap v1210: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:18:30.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:29 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/103230928' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:18:30.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:29 smithi102 ceph-mon[29849]: from='client.47213 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:18:30.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:29 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:18:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:29 smithi102 ceph-mon[41371]: pgmap v1210: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:18:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:29 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/103230928' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:18:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:29 smithi102 ceph-mon[41371]: from='client.47213 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:18:30.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:29 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:18:30.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:29 smithi146 ceph-mon[33158]: pgmap v1210: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-11T08:18:30.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:29 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/103230928' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:18:30.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:29 smithi146 ceph-mon[33158]: from='client.47213 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:18:30.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:29 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:18:30.683 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:18:30 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:18:30.730 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.729+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 <== mon.2 v1:172.21.15.102:6790/0 12 ==== osd_map(753..753 src has 1..753) v4 ==== 620+0+0 (unknown 3260725750 0 0) 0x7f3190003360 con 0x7f31a00fd540 2024-09-11T08:18:30.730 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.729+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=754}) v3 -- 0x7f31a011be90 con 0x7f31a00fd540 2024-09-11T08:18:30.736 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.735+0000 7f31977fe640 1 -- 172.21.15.102:0/103230928 <== mon.2 v1:172.21.15.102:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 91aaf327-ed21-437a-8aea-bd29873f531c v753) v1 ==== 217+0+0 (unknown 2431347437 0 0) 0x7f3190056da0 con 0x7f31a00fd540 2024-09-11T08:18:30.736 INFO:tasks.workunit.client.0.smithi102.stderr:set-quota max_bytes = 0 for pool 91aaf327-ed21-437a-8aea-bd29873f531c 2024-09-11T08:18:30.739 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.738+0000 7f31a5280640 1 -- 172.21.15.102:0/103230928 >> v1:172.21.15.102:6800/4281557748 conn(0x7f318c06fe20 legacy=0x7f318c0722d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:30.739 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.738+0000 7f31a5280640 1 -- 172.21.15.102:0/103230928 >> v1:172.21.15.102:6790/0 conn(0x7f31a00fd540 legacy=0x7f31a01040d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:30.739 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.739+0000 7f31a5280640 1 -- 172.21.15.102:0/103230928 shutdown_connections 2024-09-11T08:18:30.739 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.739+0000 7f31a5280640 1 -- 172.21.15.102:0/103230928 >> 172.21.15.102:0/103230928 conn(0x7f31a00752f0 msgr2=0x7f31a0107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:18:30.739 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.739+0000 7f31a5280640 1 -- 172.21.15.102:0/103230928 shutdown_connections 2024-09-11T08:18:30.740 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.739+0000 7f31a5280640 1 -- 172.21.15.102:0/103230928 wait complete. 2024-09-11T08:18:30.750 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph osd pool set-quota 91aaf327-ed21-437a-8aea-bd29873f531c max_objects 0 2024-09-11T08:18:30.833 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.830+0000 7fb751f5d640 1 -- 172.21.15.102:0/808171068 >> v1:172.21.15.102:6790/0 conn(0x7fb74c104250 legacy=0x7fb74c106630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:30.834 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.830+0000 7fb751f5d640 1 -- 172.21.15.102:0/808171068 shutdown_connections 2024-09-11T08:18:30.834 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.830+0000 7fb751f5d640 1 -- 172.21.15.102:0/808171068 >> 172.21.15.102:0/808171068 conn(0x7fb74c0f7020 msgr2=0x7fb74c0f9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:18:30.834 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.830+0000 7fb751f5d640 1 -- 172.21.15.102:0/808171068 shutdown_connections 2024-09-11T08:18:30.834 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.830+0000 7fb751f5d640 1 -- 172.21.15.102:0/808171068 wait complete. 2024-09-11T08:18:30.834 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.830+0000 7fb751f5d640 1 Processor -- start 2024-09-11T08:18:30.834 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.831+0000 7fb751f5d640 1 -- start start 2024-09-11T08:18:30.834 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.831+0000 7fb751f5d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb74c11beb0 con 0x7fb74c1014e0 2024-09-11T08:18:30.834 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.831+0000 7fb751f5d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb74c11d070 con 0x7fb74c104250 2024-09-11T08:18:30.834 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.831+0000 7fb751f5d640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb74c11e270 con 0x7fb74c107db0 2024-09-11T08:18:30.834 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.831+0000 7fb74b7fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb74c1014e0 0x7fb74c065730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:55558/0 (socket says 172.21.15.102:55558) 2024-09-11T08:18:30.834 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.831+0000 7fb74bfff640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7fb74c107db0 0x7fb74c11a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:41072/0 (socket says 172.21.15.102:41072) 2024-09-11T08:18:30.834 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.831+0000 7fb74b7fe640 1 -- 172.21.15.102:0/1436217500 learned_addr learned my addr 172.21.15.102:0/1436217500 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:18:30.835 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.831+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2427927200 0 0) 0x7fb74c11e270 con 0x7fb74c107db0 2024-09-11T08:18:30.835 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.831+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb734003120 con 0x7fb74c107db0 2024-09-11T08:18:30.835 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.831+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4098141249 0 0) 0x7fb74c11beb0 con 0x7fb74c1014e0 2024-09-11T08:18:30.835 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.831+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb74c11e270 con 0x7fb74c1014e0 2024-09-11T08:18:30.835 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.831+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100248015 0 0) 0x7fb74c11d070 con 0x7fb74c104250 2024-09-11T08:18:30.835 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.831+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb74c11beb0 con 0x7fb74c104250 2024-09-11T08:18:30.835 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.831+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1644587330 0 0) 0x7fb734003120 con 0x7fb74c107db0 2024-09-11T08:18:30.835 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.831+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb74c11d070 con 0x7fb74c107db0 2024-09-11T08:18:30.835 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.831+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fb744002c90 con 0x7fb74c107db0 2024-09-11T08:18:30.835 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.831+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1288020477 0 0) 0x7fb74c11e270 con 0x7fb74c1014e0 2024-09-11T08:18:30.835 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.831+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb734003120 con 0x7fb74c1014e0 2024-09-11T08:18:30.835 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.831+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fb73c0019f0 con 0x7fb74c1014e0 2024-09-11T08:18:30.836 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.832+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3418313376 0 0) 0x7fb74c11beb0 con 0x7fb74c104250 2024-09-11T08:18:30.836 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.832+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb74c11e270 con 0x7fb74c104250 2024-09-11T08:18:30.836 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.832+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fb740001200 con 0x7fb74c104250 2024-09-11T08:18:30.836 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.832+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1661123524 0 0) 0x7fb74c11d070 con 0x7fb74c107db0 2024-09-11T08:18:30.836 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.832+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 >> v1:172.21.15.146:6789/0 conn(0x7fb74c104250 legacy=0x7fb74c065e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:30.836 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.832+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 >> v1:172.21.15.102:6789/0 conn(0x7fb74c1014e0 legacy=0x7fb74c065730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:30.836 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.832+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb74c11f470 con 0x7fb74c107db0 2024-09-11T08:18:30.836 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.832+0000 7fb751f5d640 1 -- 172.21.15.102:0/1436217500 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb74c11e460 con 0x7fb74c107db0 2024-09-11T08:18:30.836 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.832+0000 7fb751f5d640 1 -- 172.21.15.102:0/1436217500 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb74c11e8b0 con 0x7fb74c107db0 2024-09-11T08:18:30.836 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.832+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7fb744001a80 con 0x7fb74c107db0 2024-09-11T08:18:30.836 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.832+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fb744005d40 con 0x7fb74c107db0 2024-09-11T08:18:30.836 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.833+0000 7fb751f5d640 1 -- 172.21.15.102:0/1436217500 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb738005040 con 0x7fb74c107db0 2024-09-11T08:18:30.837 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.833+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7fb744001630 con 0x7fb74c107db0 2024-09-11T08:18:30.837 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.834+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(753..753 src has 1..753) v4 ==== 7719+0+0 (unknown 1412822240 0 0) 0x7fb74408a6f0 con 0x7fb74c107db0 2024-09-11T08:18:30.837 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.834+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=754}) v3 -- 0x7fb74c11d070 con 0x7fb74c107db0 2024-09-11T08:18:30.838 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.838+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fb744056e30 con 0x7fb74c107db0 2024-09-11T08:18:30.996 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:30.996+0000 7fb751f5d640 1 -- 172.21.15.102:0/1436217500 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fb7380052f0 con 0x7fb74c107db0 2024-09-11T08:18:31.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:30 smithi102 ceph-mon[29849]: from='client.47213 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:18:31.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:30 smithi102 ceph-mon[29849]: osdmap e752: 8 total, 8 up, 8 in 2024-09-11T08:18:31.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:30 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/103230928' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:18:31.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:30 smithi102 ceph-mon[29849]: from='client.47213 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:18:31.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:30 smithi102 ceph-mon[41371]: from='client.47213 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:18:31.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:30 smithi102 ceph-mon[41371]: osdmap e752: 8 total, 8 up, 8 in 2024-09-11T08:18:31.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:30 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/103230928' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:18:31.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:30 smithi102 ceph-mon[41371]: from='client.47213 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:18:31.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:30 smithi146 ceph-mon[33158]: from='client.47213 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:18:31.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:30 smithi146 ceph-mon[33158]: osdmap e752: 8 total, 8 up, 8 in 2024-09-11T08:18:31.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:30 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/103230928' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:18:31.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:30 smithi146 ceph-mon[33158]: from='client.47213 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:18:31.733 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:31.732+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 <== mon.2 v1:172.21.15.102:6790/0 10 ==== osd_map(754..754 src has 1..754) v4 ==== 620+0+0 (unknown 2110276624 0 0) 0x7fb744056b70 con 0x7fb74c107db0 2024-09-11T08:18:31.733 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:31.732+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=755}) v3 -- 0x7fb734003120 con 0x7fb74c107db0 2024-09-11T08:18:31.736 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:31.735+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 <== mon.2 v1:172.21.15.102:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 91aaf327-ed21-437a-8aea-bd29873f531c v754) v1 ==== 221+0+0 (unknown 449888469 0 0) 0x7fb7440567d0 con 0x7fb74c107db0 2024-09-11T08:18:31.830 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:31.829+0000 7fb751f5d640 1 -- 172.21.15.102:0/1436217500 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fb738005a30 con 0x7fb74c107db0 2024-09-11T08:18:32.039 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.038+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 <== mon.2 v1:172.21.15.102:6790/0 12 ==== osd_map(755..755 src has 1..755) v4 ==== 620+0+0 (unknown 2445199585 0 0) 0x7fb7440567d0 con 0x7fb74c107db0 2024-09-11T08:18:32.039 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.038+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=756}) v3 -- 0x7fb74c11beb0 con 0x7fb74c107db0 2024-09-11T08:18:32.043 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.043+0000 7fb733fff640 1 -- 172.21.15.102:0/1436217500 <== mon.2 v1:172.21.15.102:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 91aaf327-ed21-437a-8aea-bd29873f531c v755) v1 ==== 221+0+0 (unknown 2592084276 0 0) 0x7fb7440569b0 con 0x7fb74c107db0 2024-09-11T08:18:32.044 INFO:tasks.workunit.client.0.smithi102.stderr:set-quota max_objects = 0 for pool 91aaf327-ed21-437a-8aea-bd29873f531c 2024-09-11T08:18:32.046 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.046+0000 7fb751f5d640 1 -- 172.21.15.102:0/1436217500 >> v1:172.21.15.102:6800/4281557748 conn(0x7fb73406fe30 legacy=0x7fb7340722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:32.047 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.046+0000 7fb751f5d640 1 -- 172.21.15.102:0/1436217500 >> v1:172.21.15.102:6790/0 conn(0x7fb74c107db0 legacy=0x7fb74c11a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:32.047 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.046+0000 7fb751f5d640 1 -- 172.21.15.102:0/1436217500 shutdown_connections 2024-09-11T08:18:32.047 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.046+0000 7fb751f5d640 1 -- 172.21.15.102:0/1436217500 >> 172.21.15.102:0/1436217500 conn(0x7fb74c0f7020 msgr2=0x7fb74c0fa470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:18:32.047 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.046+0000 7fb751f5d640 1 -- 172.21.15.102:0/1436217500 shutdown_connections 2024-09-11T08:18:32.047 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.046+0000 7fb751f5d640 1 -- 172.21.15.102:0/1436217500 wait complete. 2024-09-11T08:18:32.055 INFO:tasks.workunit.client.0.smithi102.stderr:+ wait 149586 2024-09-11T08:18:32.055 INFO:tasks.workunit.client.0.smithi102.stderr:+ '[' 0 -ne 0 ']' 2024-09-11T08:18:32.055 INFO:tasks.workunit.client.0.smithi102.stderr:+ true 2024-09-11T08:18:32.055 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 91aaf327-ed21-437a-8aea-bd29873f531c put three /etc/passwd 2024-09-11T08:18:32.084 INFO:tasks.workunit.client.0.smithi102.stderr:++ uuidgen 2024-09-11T08:18:32.085 INFO:tasks.workunit.client.0.smithi102.stderr:+ pp=06b01559-be24-4d8f-9511-86207d32787f 2024-09-11T08:18:32.085 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph osd pool create 06b01559-be24-4d8f-9511-86207d32787f 12 2024-09-11T08:18:32.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:31 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:18:32.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:31 smithi102 ceph-mon[29849]: pgmap v1212: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:18:32.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:31 smithi102 ceph-mon[29849]: from='client.47213 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:18:32.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:31 smithi102 ceph-mon[29849]: osdmap e753: 8 total, 8 up, 8 in 2024-09-11T08:18:32.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:31 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1436217500' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:18:32.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:31 smithi102 ceph-mon[29849]: from='client.47219 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:18:32.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:31 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:18:32.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:31 smithi102 ceph-mon[41371]: pgmap v1212: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:18:32.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:31 smithi102 ceph-mon[41371]: from='client.47213 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:18:32.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:31 smithi102 ceph-mon[41371]: osdmap e753: 8 total, 8 up, 8 in 2024-09-11T08:18:32.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:31 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1436217500' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:18:32.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:31 smithi102 ceph-mon[41371]: from='client.47219 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:18:32.162 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.159+0000 7fd701d54640 1 -- 172.21.15.102:0/4115800808 >> v1:172.21.15.102:6790/0 conn(0x7fd6fc107d70 legacy=0x7fd6fc10a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:32.162 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.159+0000 7fd701d54640 1 -- 172.21.15.102:0/4115800808 shutdown_connections 2024-09-11T08:18:32.162 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.159+0000 7fd701d54640 1 -- 172.21.15.102:0/4115800808 >> 172.21.15.102:0/4115800808 conn(0x7fd6fc0752f0 msgr2=0x7fd6fc0f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:18:32.162 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.159+0000 7fd701d54640 1 -- 172.21.15.102:0/4115800808 shutdown_connections 2024-09-11T08:18:32.162 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.159+0000 7fd701d54640 1 -- 172.21.15.102:0/4115800808 wait complete. 2024-09-11T08:18:32.163 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.159+0000 7fd701d54640 1 Processor -- start 2024-09-11T08:18:32.163 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.160+0000 7fd701d54640 1 -- start start 2024-09-11T08:18:32.163 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.160+0000 7fd701d54640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd6fc1078a0 con 0x7fd6fc1014a0 2024-09-11T08:18:32.163 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.160+0000 7fd701d54640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd6fc107a30 con 0x7fd6fc104210 2024-09-11T08:18:32.163 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.160+0000 7fd701d54640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd6fc120610 con 0x7fd6fc107d70 2024-09-11T08:18:32.163 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.160+0000 7fd6fb7fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fd6fc1014a0 0x7fd6fc1070a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:55600/0 (socket says 172.21.15.102:55600) 2024-09-11T08:18:32.163 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.160+0000 7fd6fb7fe640 1 -- 172.21.15.102:0/3634955619 learned_addr learned my addr 172.21.15.102:0/3634955619 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:18:32.163 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.160+0000 7fd6e3fff640 1 -- 172.21.15.102:0/3634955619 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 547385430 0 0) 0x7fd6fc1078a0 con 0x7fd6fc1014a0 2024-09-11T08:18:32.163 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.160+0000 7fd6e3fff640 1 -- 172.21.15.102:0/3634955619 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6e40034e0 con 0x7fd6fc1014a0 2024-09-11T08:18:32.163 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.160+0000 7fd6e3fff640 1 -- 172.21.15.102:0/3634955619 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2594374510 0 0) 0x7fd6fc120610 con 0x7fd6fc107d70 2024-09-11T08:18:32.163 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.160+0000 7fd6e3fff640 1 -- 172.21.15.102:0/3634955619 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6fc1078a0 con 0x7fd6fc107d70 2024-09-11T08:18:32.163 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.160+0000 7fd6e3fff640 1 -- 172.21.15.102:0/3634955619 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1116887882 0 0) 0x7fd6fc1078a0 con 0x7fd6fc107d70 2024-09-11T08:18:32.163 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.160+0000 7fd6e3fff640 1 -- 172.21.15.102:0/3634955619 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd6fc120610 con 0x7fd6fc107d70 2024-09-11T08:18:32.164 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.160+0000 7fd6e3fff640 1 -- 172.21.15.102:0/3634955619 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 709693611 0 0) 0x7fd6e40034e0 con 0x7fd6fc1014a0 2024-09-11T08:18:32.164 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.160+0000 7fd6e3fff640 1 -- 172.21.15.102:0/3634955619 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd6fc1078a0 con 0x7fd6fc1014a0 2024-09-11T08:18:32.164 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.160+0000 7fd6e3fff640 1 -- 172.21.15.102:0/3634955619 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fd6f4002de0 con 0x7fd6fc107d70 2024-09-11T08:18:32.164 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.160+0000 7fd6e3fff640 1 -- 172.21.15.102:0/3634955619 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fd6ec001940 con 0x7fd6fc1014a0 2024-09-11T08:18:32.164 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.161+0000 7fd6e3fff640 1 -- 172.21.15.102:0/3634955619 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114724633 0 0) 0x7fd6fc107a30 con 0x7fd6fc104210 2024-09-11T08:18:32.164 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.161+0000 7fd6e3fff640 1 -- 172.21.15.102:0/3634955619 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6e40034e0 con 0x7fd6fc104210 2024-09-11T08:18:32.164 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.161+0000 7fd6e3fff640 1 -- 172.21.15.102:0/3634955619 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1086268971 0 0) 0x7fd6fc120610 con 0x7fd6fc107d70 2024-09-11T08:18:32.164 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.161+0000 7fd6e3fff640 1 -- 172.21.15.102:0/3634955619 >> v1:172.21.15.146:6789/0 conn(0x7fd6fc104210 legacy=0x7fd6fc1197a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:32.164 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.161+0000 7fd6e3fff640 1 -- 172.21.15.102:0/3634955619 >> v1:172.21.15.102:6789/0 conn(0x7fd6fc1014a0 legacy=0x7fd6fc1070a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:32.164 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.161+0000 7fd6e3fff640 1 -- 172.21.15.102:0/3634955619 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc1217b0 con 0x7fd6fc107d70 2024-09-11T08:18:32.164 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.161+0000 7fd701d54640 1 -- 172.21.15.102:0/3634955619 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6fc120800 con 0x7fd6fc107d70 2024-09-11T08:18:32.164 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.161+0000 7fd701d54640 1 -- 172.21.15.102:0/3634955619 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6fc120cd0 con 0x7fd6fc107d70 2024-09-11T08:18:32.164 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.161+0000 7fd6e3fff640 1 -- 172.21.15.102:0/3634955619 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7fd6f40030a0 con 0x7fd6fc107d70 2024-09-11T08:18:32.165 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.161+0000 7fd6e3fff640 1 -- 172.21.15.102:0/3634955619 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fd6f4003900 con 0x7fd6fc107d70 2024-09-11T08:18:32.165 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.162+0000 7fd701d54640 1 -- 172.21.15.102:0/3634955619 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e8005010 con 0x7fd6fc107d70 2024-09-11T08:18:32.165 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.162+0000 7fd6e3fff640 1 -- 172.21.15.102:0/3634955619 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7fd6f4002520 con 0x7fd6fc107d70 2024-09-11T08:18:32.165 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.163+0000 7fd6e3fff640 1 -- 172.21.15.102:0/3634955619 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(755..755 src has 1..755) v4 ==== 7719+0+0 (unknown 3676376096 0 0) 0x7fd6f4088200 con 0x7fd6fc107d70 2024-09-11T08:18:32.167 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.166+0000 7fd6e3fff640 1 -- 172.21.15.102:0/3634955619 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fd6f40548e0 con 0x7fd6fc107d70 2024-09-11T08:18:32.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:31 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:18:32.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:31 smithi146 ceph-mon[33158]: pgmap v1212: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:18:32.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:31 smithi146 ceph-mon[33158]: from='client.47213 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:18:32.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:31 smithi146 ceph-mon[33158]: osdmap e753: 8 total, 8 up, 8 in 2024-09-11T08:18:32.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:31 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1436217500' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:18:32.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:31 smithi146 ceph-mon[33158]: from='client.47219 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:18:32.322 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:32.321+0000 7fd701d54640 1 -- 172.21.15.102:0/3634955619 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pg_num": 12} v 0) v1 -- 0x7fd6e8005840 con 0x7fd6fc107d70 2024-09-11T08:18:33.050 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.049+0000 7fd6e3fff640 1 -- 172.21.15.102:0/3634955619 <== mon.2 v1:172.21.15.102:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pg_num": 12}]=0 pool '06b01559-be24-4d8f-9511-86207d32787f' created v756) v1 ==== 176+0+0 (unknown 4288049590 0 0) 0x7fd6f4001550 con 0x7fd6fc107d70 2024-09-11T08:18:33.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:32 smithi102 ceph-mon[29849]: from='client.47219 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:18:33.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:32 smithi102 ceph-mon[29849]: osdmap e754: 8 total, 8 up, 8 in 2024-09-11T08:18:33.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1436217500' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:18:33.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:32 smithi102 ceph-mon[29849]: from='client.47219 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:18:33.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:32 smithi102 ceph-mon[29849]: pool '91aaf327-ed21-437a-8aea-bd29873f531c' no longer out of quota; removing NO_QUOTA flag 2024-09-11T08:18:33.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:32 smithi102 ceph-mon[29849]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:18:33.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:32 smithi102 ceph-mon[29849]: from='client.47219 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:18:33.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:32 smithi102 ceph-mon[29849]: osdmap e755: 8 total, 8 up, 8 in 2024-09-11T08:18:33.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:32 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3634955619' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pg_num": 12}]: dispatch 2024-09-11T08:18:33.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:32 smithi102 ceph-mon[29849]: from='client.47234 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pg_num": 12}]: dispatch 2024-09-11T08:18:33.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:32 smithi102 ceph-mon[41371]: from='client.47219 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:18:33.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:32 smithi102 ceph-mon[41371]: osdmap e754: 8 total, 8 up, 8 in 2024-09-11T08:18:33.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1436217500' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:18:33.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:32 smithi102 ceph-mon[41371]: from='client.47219 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:18:33.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:32 smithi102 ceph-mon[41371]: pool '91aaf327-ed21-437a-8aea-bd29873f531c' no longer out of quota; removing NO_QUOTA flag 2024-09-11T08:18:33.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:32 smithi102 ceph-mon[41371]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:18:33.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:32 smithi102 ceph-mon[41371]: from='client.47219 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:18:33.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:32 smithi102 ceph-mon[41371]: osdmap e755: 8 total, 8 up, 8 in 2024-09-11T08:18:33.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:32 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3634955619' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pg_num": 12}]: dispatch 2024-09-11T08:18:33.147 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:32 smithi102 ceph-mon[41371]: from='client.47234 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pg_num": 12}]: dispatch 2024-09-11T08:18:33.147 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.145+0000 7fd701d54640 1 -- 172.21.15.102:0/3634955619 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pg_num": 12} v 0) v1 -- 0x7fd6e80052c0 con 0x7fd6fc107d70 2024-09-11T08:18:33.147 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.146+0000 7fd6e3fff640 1 -- 172.21.15.102:0/3634955619 <== mon.2 v1:172.21.15.102:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pg_num": 12}]=0 pool '06b01559-be24-4d8f-9511-86207d32787f' already exists v756) v1 ==== 183+0+0 (unknown 1427293846 0 0) 0x7fd6f4054280 con 0x7fd6fc107d70 2024-09-11T08:18:33.148 INFO:tasks.workunit.client.0.smithi102.stderr:pool '06b01559-be24-4d8f-9511-86207d32787f' already exists 2024-09-11T08:18:33.148 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.147+0000 7fd701d54640 1 -- 172.21.15.102:0/3634955619 >> v1:172.21.15.102:6800/4281557748 conn(0x7fd6e406fda0 legacy=0x7fd6e4072250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:33.148 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.147+0000 7fd701d54640 1 -- 172.21.15.102:0/3634955619 >> v1:172.21.15.102:6790/0 conn(0x7fd6fc107d70 legacy=0x7fd6fc11ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:33.149 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.148+0000 7fd701d54640 1 -- 172.21.15.102:0/3634955619 shutdown_connections 2024-09-11T08:18:33.149 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.148+0000 7fd701d54640 1 -- 172.21.15.102:0/3634955619 >> 172.21.15.102:0/3634955619 conn(0x7fd6fc0752f0 msgr2=0x7fd6fc0f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:18:33.149 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.148+0000 7fd701d54640 1 -- 172.21.15.102:0/3634955619 shutdown_connections 2024-09-11T08:18:33.149 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.148+0000 7fd701d54640 1 -- 172.21.15.102:0/3634955619 wait complete. 2024-09-11T08:18:33.157 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph osd pool application enable 06b01559-be24-4d8f-9511-86207d32787f rados 2024-09-11T08:18:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:32 smithi146 ceph-mon[33158]: from='client.47219 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:18:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:32 smithi146 ceph-mon[33158]: osdmap e754: 8 total, 8 up, 8 in 2024-09-11T08:18:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1436217500' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:18:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:32 smithi146 ceph-mon[33158]: from='client.47219 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:18:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:32 smithi146 ceph-mon[33158]: pool '91aaf327-ed21-437a-8aea-bd29873f531c' no longer out of quota; removing NO_QUOTA flag 2024-09-11T08:18:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:32 smithi146 ceph-mon[33158]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:18:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:32 smithi146 ceph-mon[33158]: from='client.47219 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:18:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:32 smithi146 ceph-mon[33158]: osdmap e755: 8 total, 8 up, 8 in 2024-09-11T08:18:33.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:32 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3634955619' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pg_num": 12}]: dispatch 2024-09-11T08:18:33.185 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:32 smithi146 ceph-mon[33158]: from='client.47234 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pg_num": 12}]: dispatch 2024-09-11T08:18:33.236 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.233+0000 7faf79e89640 1 -- 172.21.15.102:0/1416944537 >> v1:172.21.15.102:6790/0 conn(0x7faf74104230 legacy=0x7faf74106610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:33.236 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.233+0000 7faf79e89640 1 -- 172.21.15.102:0/1416944537 shutdown_connections 2024-09-11T08:18:33.236 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.233+0000 7faf79e89640 1 -- 172.21.15.102:0/1416944537 >> 172.21.15.102:0/1416944537 conn(0x7faf740f6fe0 msgr2=0x7faf740f93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:18:33.236 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.233+0000 7faf79e89640 1 -- 172.21.15.102:0/1416944537 shutdown_connections 2024-09-11T08:18:33.236 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.233+0000 7faf79e89640 1 -- 172.21.15.102:0/1416944537 wait complete. 2024-09-11T08:18:33.236 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.233+0000 7faf79e89640 1 Processor -- start 2024-09-11T08:18:33.236 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.233+0000 7faf79e89640 1 -- start start 2024-09-11T08:18:33.236 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.234+0000 7faf79e89640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7faf7411bf50 con 0x7faf741014c0 2024-09-11T08:18:33.236 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.234+0000 7faf79e89640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7faf7411d150 con 0x7faf74107d90 2024-09-11T08:18:33.236 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.234+0000 7faf79e89640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7faf7411e350 con 0x7faf74104230 2024-09-11T08:18:33.237 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.234+0000 7faf737fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7faf741014c0 0x7faf74065c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:55628/0 (socket says 172.21.15.102:55628) 2024-09-11T08:18:33.237 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.234+0000 7faf72ffd640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7faf74104230 0x7faf74066360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:41136/0 (socket says 172.21.15.102:41136) 2024-09-11T08:18:33.237 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.234+0000 7faf737fe640 1 -- 172.21.15.102:0/23596467 learned_addr learned my addr 172.21.15.102:0/23596467 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:18:33.237 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.234+0000 7faf5bfff640 1 -- 172.21.15.102:0/23596467 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339590965 0 0) 0x7faf7411e350 con 0x7faf74104230 2024-09-11T08:18:33.237 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.234+0000 7faf5bfff640 1 -- 172.21.15.102:0/23596467 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf5c0034e0 con 0x7faf74104230 2024-09-11T08:18:33.237 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.234+0000 7faf5bfff640 1 -- 172.21.15.102:0/23596467 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3124884655 0 0) 0x7faf7411bf50 con 0x7faf741014c0 2024-09-11T08:18:33.237 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.234+0000 7faf5bfff640 1 -- 172.21.15.102:0/23596467 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf7411e350 con 0x7faf741014c0 2024-09-11T08:18:33.237 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.234+0000 7faf5bfff640 1 -- 172.21.15.102:0/23596467 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3591575839 0 0) 0x7faf7411d150 con 0x7faf74107d90 2024-09-11T08:18:33.237 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.234+0000 7faf5bfff640 1 -- 172.21.15.102:0/23596467 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf7411bf50 con 0x7faf74107d90 2024-09-11T08:18:33.238 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.234+0000 7faf5bfff640 1 -- 172.21.15.102:0/23596467 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4089028002 0 0) 0x7faf7411e350 con 0x7faf741014c0 2024-09-11T08:18:33.238 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.234+0000 7faf5bfff640 1 -- 172.21.15.102:0/23596467 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faf7411d150 con 0x7faf741014c0 2024-09-11T08:18:33.238 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.234+0000 7faf5bfff640 1 -- 172.21.15.102:0/23596467 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7faf64001cf0 con 0x7faf741014c0 2024-09-11T08:18:33.238 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.234+0000 7faf5bfff640 1 -- 172.21.15.102:0/23596467 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2668481261 0 0) 0x7faf5c0034e0 con 0x7faf74104230 2024-09-11T08:18:33.238 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.234+0000 7faf5bfff640 1 -- 172.21.15.102:0/23596467 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faf7411e350 con 0x7faf74104230 2024-09-11T08:18:33.238 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.234+0000 7faf5bfff640 1 -- 172.21.15.102:0/23596467 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7faf68002910 con 0x7faf74104230 2024-09-11T08:18:33.238 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.234+0000 7faf5bfff640 1 -- 172.21.15.102:0/23596467 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 774094928 0 0) 0x7faf7411d150 con 0x7faf741014c0 2024-09-11T08:18:33.238 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.234+0000 7faf5bfff640 1 -- 172.21.15.102:0/23596467 >> v1:172.21.15.102:6790/0 conn(0x7faf74104230 legacy=0x7faf74066360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:33.238 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.234+0000 7faf5bfff640 1 -- 172.21.15.102:0/23596467 >> v1:172.21.15.146:6789/0 conn(0x7faf74107d90 legacy=0x7faf7411a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:33.238 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.234+0000 7faf5bfff640 1 -- 172.21.15.102:0/23596467 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7411f550 con 0x7faf741014c0 2024-09-11T08:18:33.239 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.234+0000 7faf79e89640 1 -- 172.21.15.102:0/23596467 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf7411e540 con 0x7faf741014c0 2024-09-11T08:18:33.239 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.235+0000 7faf79e89640 1 -- 172.21.15.102:0/23596467 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7faf7411e990 con 0x7faf741014c0 2024-09-11T08:18:33.239 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.235+0000 7faf5bfff640 1 -- 172.21.15.102:0/23596467 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7faf640027d0 con 0x7faf741014c0 2024-09-11T08:18:33.239 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.235+0000 7faf5bfff640 1 -- 172.21.15.102:0/23596467 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7faf64005e10 con 0x7faf741014c0 2024-09-11T08:18:33.239 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.235+0000 7faf79e89640 1 -- 172.21.15.102:0/23596467 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf60005010 con 0x7faf741014c0 2024-09-11T08:18:33.240 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.236+0000 7faf5bfff640 1 -- 172.21.15.102:0/23596467 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7faf64002380 con 0x7faf741014c0 2024-09-11T08:18:33.240 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.236+0000 7faf5bfff640 1 -- 172.21.15.102:0/23596467 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(756..756 src has 1..756) v4 ==== 8094+0+0 (unknown 1605853318 0 0) 0x7faf6408aa60 con 0x7faf741014c0 2024-09-11T08:18:33.241 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.241+0000 7faf5bfff640 1 -- 172.21.15.102:0/23596467 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7faf64057020 con 0x7faf741014c0 2024-09-11T08:18:33.396 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:33.396+0000 7faf79e89640 1 -- 172.21.15.102:0/23596467 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "osd pool application enable", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "app": "rados"} v 0) v1 -- 0x7faf600052c0 con 0x7faf741014c0 2024-09-11T08:18:34.050 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:34.049+0000 7faf5bfff640 1 -- 172.21.15.102:0/23596467 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "app": "rados"}]=0 enabled application 'rados' on pool '06b01559-be24-4d8f-9511-86207d32787f' v757) v1 ==== 213+0+0 (unknown 4202058445 0 0) 0x7faf640569c0 con 0x7faf741014c0 2024-09-11T08:18:34.085 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:33 smithi146 ceph-mon[33158]: pgmap v1216: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:18:34.085 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:33 smithi146 ceph-mon[33158]: from='client.47234 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pg_num": 12}]': finished 2024-09-11T08:18:34.085 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:33 smithi146 ceph-mon[33158]: osdmap e756: 8 total, 8 up, 8 in 2024-09-11T08:18:34.085 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3634955619' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pg_num": 12}]: dispatch 2024-09-11T08:18:34.085 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:33 smithi146 ceph-mon[33158]: from='client.47234 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pg_num": 12}]: dispatch 2024-09-11T08:18:34.086 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:33 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/23596467' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "app": "rados"}]: dispatch 2024-09-11T08:18:34.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:33 smithi102 ceph-mon[41371]: pgmap v1216: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:18:34.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:33 smithi102 ceph-mon[41371]: from='client.47234 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pg_num": 12}]': finished 2024-09-11T08:18:34.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:33 smithi102 ceph-mon[41371]: osdmap e756: 8 total, 8 up, 8 in 2024-09-11T08:18:34.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3634955619' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pg_num": 12}]: dispatch 2024-09-11T08:18:34.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:33 smithi102 ceph-mon[41371]: from='client.47234 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pg_num": 12}]: dispatch 2024-09-11T08:18:34.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:33 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/23596467' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "app": "rados"}]: dispatch 2024-09-11T08:18:34.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:33 smithi102 ceph-mon[29849]: pgmap v1216: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:18:34.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:33 smithi102 ceph-mon[29849]: from='client.47234 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pg_num": 12}]': finished 2024-09-11T08:18:34.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:33 smithi102 ceph-mon[29849]: osdmap e756: 8 total, 8 up, 8 in 2024-09-11T08:18:34.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3634955619' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pg_num": 12}]: dispatch 2024-09-11T08:18:34.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:33 smithi102 ceph-mon[29849]: from='client.47234 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pg_num": 12}]: dispatch 2024-09-11T08:18:34.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:33 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/23596467' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "app": "rados"}]: dispatch 2024-09-11T08:18:34.150 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:34.150+0000 7faf79e89640 1 -- 172.21.15.102:0/23596467 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "osd pool application enable", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "app": "rados"} v 0) v1 -- 0x7faf60001090 con 0x7faf741014c0 2024-09-11T08:18:34.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:18:34 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:18:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:18:35.063 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.062+0000 7faf5bfff640 1 -- 172.21.15.102:0/23596467 <== mon.0 v1:172.21.15.102:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "app": "rados"}]=0 enabled application 'rados' on pool '06b01559-be24-4d8f-9511-86207d32787f' v758) v1 ==== 213+0+0 (unknown 3021228519 0 0) 0x7faf64056ba0 con 0x7faf741014c0 2024-09-11T08:18:35.063 INFO:tasks.workunit.client.0.smithi102.stderr:enabled application 'rados' on pool '06b01559-be24-4d8f-9511-86207d32787f' 2024-09-11T08:18:35.065 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.065+0000 7faf79e89640 1 -- 172.21.15.102:0/23596467 >> v1:172.21.15.102:6800/4281557748 conn(0x7faf5c06fd30 legacy=0x7faf5c0721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:35.065 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.065+0000 7faf79e89640 1 -- 172.21.15.102:0/23596467 >> v1:172.21.15.102:6789/0 conn(0x7faf741014c0 legacy=0x7faf74065c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:35.066 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.065+0000 7faf79e89640 1 -- 172.21.15.102:0/23596467 shutdown_connections 2024-09-11T08:18:35.066 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.065+0000 7faf79e89640 1 -- 172.21.15.102:0/23596467 >> 172.21.15.102:0/23596467 conn(0x7faf740f6fe0 msgr2=0x7faf740fbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:18:35.066 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.065+0000 7faf79e89640 1 -- 172.21.15.102:0/23596467 shutdown_connections 2024-09-11T08:18:35.066 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.065+0000 7faf79e89640 1 -- 172.21.15.102:0/23596467 wait complete. 2024-09-11T08:18:35.075 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph osd pool set-quota 06b01559-be24-4d8f-9511-86207d32787f max_objects 10 2024-09-11T08:18:35.159 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.154+0000 7fc7bb99d640 1 -- 172.21.15.102:0/3675180789 >> v1:172.21.15.102:6790/0 conn(0x7fc7b410f770 legacy=0x7fc7b4111b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:35.159 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.154+0000 7fc7bb99d640 1 -- 172.21.15.102:0/3675180789 shutdown_connections 2024-09-11T08:18:35.160 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.154+0000 7fc7bb99d640 1 -- 172.21.15.102:0/3675180789 >> 172.21.15.102:0/3675180789 conn(0x7fc7b40f8130 msgr2=0x7fc7b40fa540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:18:35.160 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.154+0000 7fc7bb99d640 1 -- 172.21.15.102:0/3675180789 shutdown_connections 2024-09-11T08:18:35.160 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.154+0000 7fc7bb99d640 1 -- 172.21.15.102:0/3675180789 wait complete. 2024-09-11T08:18:35.160 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.155+0000 7fc7bb99d640 1 Processor -- start 2024-09-11T08:18:35.160 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.155+0000 7fc7bb99d640 1 -- start start 2024-09-11T08:18:35.160 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.155+0000 7fc7bb99d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc7b41225d0 con 0x7fc7b410f770 2024-09-11T08:18:35.160 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.155+0000 7fc7bb99d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc7b4123790 con 0x7fc7b40716e0 2024-09-11T08:18:35.160 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.155+0000 7fc7bb99d640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc7b4124990 con 0x7fc7b40701c0 2024-09-11T08:18:35.160 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.155+0000 7fc7b9739640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7fc7b40701c0 0x7fc7b410bb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:41146/0 (socket says 172.21.15.102:41146) 2024-09-11T08:18:35.160 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.155+0000 7fc7b9739640 1 -- 172.21.15.102:0/1377155941 learned_addr learned my addr 172.21.15.102:0/1377155941 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:18:35.160 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.155+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1197982766 0 0) 0x7fc7b4124990 con 0x7fc7b40701c0 2024-09-11T08:18:35.160 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.155+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7a00034e0 con 0x7fc7b40701c0 2024-09-11T08:18:35.161 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.155+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763163858 0 0) 0x7fc7b41225d0 con 0x7fc7b410f770 2024-09-11T08:18:35.161 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.155+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7b4124990 con 0x7fc7b410f770 2024-09-11T08:18:35.161 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.156+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1079339662 0 0) 0x7fc7b4123790 con 0x7fc7b40716e0 2024-09-11T08:18:35.161 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.156+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7b41225d0 con 0x7fc7b40716e0 2024-09-11T08:18:35.161 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.156+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1392937646 0 0) 0x7fc7b4124990 con 0x7fc7b410f770 2024-09-11T08:18:35.161 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.156+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc7b4123790 con 0x7fc7b410f770 2024-09-11T08:18:35.161 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.156+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 629231957 0 0) 0x7fc7a00034e0 con 0x7fc7b40701c0 2024-09-11T08:18:35.161 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.156+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc7b4124990 con 0x7fc7b40701c0 2024-09-11T08:18:35.161 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.156+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fc7ac002420 con 0x7fc7b410f770 2024-09-11T08:18:35.161 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.156+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fc7b0001970 con 0x7fc7b40701c0 2024-09-11T08:18:35.162 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.156+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3992483632 0 0) 0x7fc7b41225d0 con 0x7fc7b40716e0 2024-09-11T08:18:35.162 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.156+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc7a00034e0 con 0x7fc7b40716e0 2024-09-11T08:18:35.162 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.156+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fc7a40018c0 con 0x7fc7b40716e0 2024-09-11T08:18:35.162 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.156+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 269527025 0 0) 0x7fc7b4123790 con 0x7fc7b410f770 2024-09-11T08:18:35.162 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.156+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 >> v1:172.21.15.102:6790/0 conn(0x7fc7b40701c0 legacy=0x7fc7b410bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:35.162 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.156+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 >> v1:172.21.15.146:6789/0 conn(0x7fc7b40716e0 legacy=0x7fc7b410c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:35.162 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.156+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b4125b90 con 0x7fc7b410f770 2024-09-11T08:18:35.162 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.156+0000 7fc7bb99d640 1 -- 172.21.15.102:0/1377155941 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b4124b80 con 0x7fc7b410f770 2024-09-11T08:18:35.162 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.156+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7fc7ac002610 con 0x7fc7b410f770 2024-09-11T08:18:35.162 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.156+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fc7ac003920 con 0x7fc7b410f770 2024-09-11T08:18:35.162 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.156+0000 7fc7bb99d640 1 -- 172.21.15.102:0/1377155941 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b4124fd0 con 0x7fc7b410f770 2024-09-11T08:18:35.162 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.157+0000 7fc7bb99d640 1 -- 172.21.15.102:0/1377155941 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b40711f0 con 0x7fc7b410f770 2024-09-11T08:18:35.162 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.157+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7fc7ac001bc0 con 0x7fc7b410f770 2024-09-11T08:18:35.163 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.158+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(758..758 src has 1..758) v4 ==== 8107+0+0 (unknown 2122884530 0 0) 0x7fc7ac0882d0 con 0x7fc7b410f770 2024-09-11T08:18:35.165 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.164+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fc7ac054820 con 0x7fc7b410f770 2024-09-11T08:18:35.323 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:35.323+0000 7fc7bb99d640 1 -- 172.21.15.102:0/1377155941 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fc7b40646c0 con 0x7fc7b410f770 2024-09-11T08:18:35.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/23596467' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "app": "rados"}]': finished 2024-09-11T08:18:35.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:35 smithi102 ceph-mon[41371]: osdmap e757: 8 total, 8 up, 8 in 2024-09-11T08:18:35.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:35 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/23596467' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "app": "rados"}]: dispatch 2024-09-11T08:18:35.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:35 smithi102 ceph-mon[41371]: pgmap v1219: 188 pgs: 12 unknown, 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:18:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/23596467' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "app": "rados"}]': finished 2024-09-11T08:18:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:35 smithi102 ceph-mon[29849]: osdmap e757: 8 total, 8 up, 8 in 2024-09-11T08:18:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:35 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/23596467' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "app": "rados"}]: dispatch 2024-09-11T08:18:35.396 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:35 smithi102 ceph-mon[29849]: pgmap v1219: 188 pgs: 12 unknown, 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:18:35.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/23596467' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "app": "rados"}]': finished 2024-09-11T08:18:35.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:35 smithi146 ceph-mon[33158]: osdmap e757: 8 total, 8 up, 8 in 2024-09-11T08:18:35.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:35 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/23596467' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "app": "rados"}]: dispatch 2024-09-11T08:18:35.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:35 smithi146 ceph-mon[33158]: pgmap v1219: 188 pgs: 12 unknown, 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T08:18:36.058 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:36.058+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 <== mon.0 v1:172.21.15.102:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 06b01559-be24-4d8f-9511-86207d32787f v759) v1 ==== 223+0+0 (unknown 2711028202 0 0) 0x7fc7ac0541c0 con 0x7fc7b410f770 2024-09-11T08:18:36.155 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:36.154+0000 7fc7bb99d640 1 -- 172.21.15.102:0/1377155941 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fc7b4114fc0 con 0x7fc7b410f770 2024-09-11T08:18:36.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:36 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/23596467' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "app": "rados"}]': finished 2024-09-11T08:18:36.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:36 smithi102 ceph-mon[29849]: osdmap e758: 8 total, 8 up, 8 in 2024-09-11T08:18:36.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:36 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1377155941' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:18:36.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:36 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/23596467' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "app": "rados"}]': finished 2024-09-11T08:18:36.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:36 smithi102 ceph-mon[41371]: osdmap e758: 8 total, 8 up, 8 in 2024-09-11T08:18:36.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:36 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1377155941' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:18:36.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:36 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/23596467' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "app": "rados"}]': finished 2024-09-11T08:18:36.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:36 smithi146 ceph-mon[33158]: osdmap e758: 8 total, 8 up, 8 in 2024-09-11T08:18:36.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:36 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1377155941' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:18:37.075 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:37.074+0000 7fc7a9ffb640 1 -- 172.21.15.102:0/1377155941 <== mon.0 v1:172.21.15.102:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 06b01559-be24-4d8f-9511-86207d32787f v760) v1 ==== 223+0+0 (unknown 458672937 0 0) 0x7fc7ac0543a0 con 0x7fc7b410f770 2024-09-11T08:18:37.076 INFO:tasks.workunit.client.0.smithi102.stderr:set-quota max_objects = 10 for pool 06b01559-be24-4d8f-9511-86207d32787f 2024-09-11T08:18:37.078 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:37.077+0000 7fc7bb99d640 1 -- 172.21.15.102:0/1377155941 >> v1:172.21.15.102:6800/4281557748 conn(0x7fc7a006ffb0 legacy=0x7fc7a0072460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:37.078 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:37.077+0000 7fc7bb99d640 1 -- 172.21.15.102:0/1377155941 >> v1:172.21.15.102:6789/0 conn(0x7fc7b410f770 legacy=0x7fc7b4120cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:18:37.078 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:37.077+0000 7fc7bb99d640 1 -- 172.21.15.102:0/1377155941 shutdown_connections 2024-09-11T08:18:37.078 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:37.077+0000 7fc7bb99d640 1 -- 172.21.15.102:0/1377155941 >> 172.21.15.102:0/1377155941 conn(0x7fc7b40f8130 msgr2=0x7fc7b40f9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:18:37.078 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:37.078+0000 7fc7bb99d640 1 -- 172.21.15.102:0/1377155941 shutdown_connections 2024-09-11T08:18:37.078 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:18:37.078+0000 7fc7bb99d640 1 -- 172.21.15.102:0/1377155941 wait complete. 2024-09-11T08:18:37.089 INFO:tasks.workunit.client.0.smithi102.stderr:+ sleep 30 2024-09-11T08:18:37.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1377155941' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:18:37.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:37 smithi102 ceph-mon[29849]: osdmap e759: 8 total, 8 up, 8 in 2024-09-11T08:18:37.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1377155941' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:18:37.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:37 smithi102 ceph-mon[29849]: pgmap v1222: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-11T08:18:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1377155941' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:18:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:37 smithi102 ceph-mon[41371]: osdmap e759: 8 total, 8 up, 8 in 2024-09-11T08:18:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1377155941' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:18:37.396 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:37 smithi102 ceph-mon[41371]: pgmap v1222: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-11T08:18:37.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1377155941' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:18:37.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:37 smithi146 ceph-mon[33158]: osdmap e759: 8 total, 8 up, 8 in 2024-09-11T08:18:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1377155941' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "field": "max_objects", "val": "10"}]: dispatch 2024-09-11T08:18:37.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:37 smithi146 ceph-mon[33158]: pgmap v1222: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-11T08:18:38.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/1377155941' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:18:38.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:38 smithi102 ceph-mon[41371]: osdmap e760: 8 total, 8 up, 8 in 2024-09-11T08:18:38.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/1377155941' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:18:38.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:38 smithi102 ceph-mon[29849]: osdmap e760: 8 total, 8 up, 8 in 2024-09-11T08:18:38.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/1377155941' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "field": "max_objects", "val": "10"}]': finished 2024-09-11T08:18:38.434 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:38 smithi146 ceph-mon[33158]: osdmap e760: 8 total, 8 up, 8 in 2024-09-11T08:18:39.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:39 smithi102 ceph-mon[29849]: pgmap v1224: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.7 KiB/s wr, 1 op/s 2024-09-11T08:18:39.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:39 smithi102 ceph-mon[41371]: pgmap v1224: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.7 KiB/s wr, 1 op/s 2024-09-11T08:18:39.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:39 smithi146 ceph-mon[33158]: pgmap v1224: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 5.7 KiB/s wr, 1 op/s 2024-09-11T08:18:40.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:18:39 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:18:39] "GET /metrics HTTP/1.1" 200 226235 "" "Prometheus/2.43.0" 2024-09-11T08:18:40.683 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:18:40 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:18:42.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:41 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:18:42.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:41 smithi102 ceph-mon[41371]: pgmap v1225: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-11T08:18:42.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:41 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:18:42.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:41 smithi102 ceph-mon[29849]: pgmap v1225: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-11T08:18:42.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:41 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:18:42.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:41 smithi146 ceph-mon[33158]: pgmap v1225: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-11T08:18:44.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:43 smithi102 ceph-mon[41371]: pgmap v1226: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.4 KiB/s wr, 1 op/s 2024-09-11T08:18:44.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:43 smithi102 ceph-mon[29849]: pgmap v1226: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.4 KiB/s wr, 1 op/s 2024-09-11T08:18:44.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:43 smithi146 ceph-mon[33158]: pgmap v1226: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.4 KiB/s wr, 1 op/s 2024-09-11T08:18:44.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:18:44 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:18:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:18:45.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:45 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:18:45.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:45 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:18:45.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:45 smithi146 ceph-mon[33158]: pgmap v1227: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 596 B/s rd, 0 op/s 2024-09-11T08:18:45.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:45 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:18:45.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:45 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:18:45.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:45 smithi102 ceph-mon[41371]: pgmap v1227: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 596 B/s rd, 0 op/s 2024-09-11T08:18:45.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:45 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:18:45.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:45 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:18:45.896 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:45 smithi102 ceph-mon[29849]: pgmap v1227: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 596 B/s rd, 0 op/s 2024-09-11T08:18:48.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:47 smithi102 ceph-mon[29849]: pgmap v1228: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:18:48.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:47 smithi102 ceph-mon[41371]: pgmap v1228: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:18:48.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:47 smithi146 ceph-mon[33158]: pgmap v1228: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:18:50.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:49 smithi102 ceph-mon[29849]: pgmap v1229: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 884 B/s rd, 0 op/s 2024-09-11T08:18:50.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:49 smithi102 ceph-mon[41371]: pgmap v1229: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 884 B/s rd, 0 op/s 2024-09-11T08:18:50.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:18:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:18:49] "GET /metrics HTTP/1.1" 200 228716 "" "Prometheus/2.43.0" 2024-09-11T08:18:50.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:49 smithi146 ceph-mon[33158]: pgmap v1229: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 884 B/s rd, 0 op/s 2024-09-11T08:18:50.683 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:18:50 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:18:51.047 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:50 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:18:51.047 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:50 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:18:51.047 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:50 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:18:51.047 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:50 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:18:51.047 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:50 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:18:51.047 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:50 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:18:51.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:50 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:18:51.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:50 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:18:51.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:50 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:18:52.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:51 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:18:52.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:51 smithi102 ceph-mon[29849]: pgmap v1230: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:18:52.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:51 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:18:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:51 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:18:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:51 smithi102 ceph-mon[41371]: pgmap v1230: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:18:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:51 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:18:52.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:51 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:18:52.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:51 smithi146 ceph-mon[33158]: pgmap v1230: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:18:52.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:51 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:18:54.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:53 smithi102 ceph-mon[29849]: pgmap v1231: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:18:54.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:53 smithi102 ceph-mon[41371]: pgmap v1231: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:18:54.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:53 smithi146 ceph-mon[33158]: pgmap v1231: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:18:54.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:18:54 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:18:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:18:56.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:55 smithi102 ceph-mon[41371]: pgmap v1232: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:18:56.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:55 smithi102 ceph-mon[29849]: pgmap v1232: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:18:56.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:55 smithi146 ceph-mon[33158]: pgmap v1232: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:18:58.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:57 smithi102 ceph-mon[29849]: pgmap v1233: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:18:58.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:57 smithi102 ceph-mon[41371]: pgmap v1233: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:18:58.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:57 smithi146 ceph-mon[33158]: pgmap v1233: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:19:00.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:59 smithi102 ceph-mon[41371]: pgmap v1234: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:00.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:18:59 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:19:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:59 smithi102 ceph-mon[29849]: pgmap v1234: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:18:59 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:19:00.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:18:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:18:59] "GET /metrics HTTP/1.1" 200 228683 "" "Prometheus/2.43.0" 2024-09-11T08:19:00.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:59 smithi146 ceph-mon[33158]: pgmap v1234: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:00.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:18:59 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:19:00.683 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:19:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:19:02.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:01 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:19:02.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:01 smithi102 ceph-mon[41371]: pgmap v1235: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:19:02.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:01 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:19:02.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:01 smithi102 ceph-mon[29849]: pgmap v1235: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:19:02.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:01 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:19:02.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:01 smithi146 ceph-mon[33158]: pgmap v1235: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:19:04.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:03 smithi102 ceph-mon[29849]: pgmap v1236: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:04.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:03 smithi102 ceph-mon[41371]: pgmap v1236: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:04.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:03 smithi146 ceph-mon[33158]: pgmap v1236: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:04.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:19:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:19:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:19:06.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:05 smithi102 ceph-mon[41371]: pgmap v1237: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:06.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:05 smithi102 ceph-mon[29849]: pgmap v1237: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:06.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:05 smithi146 ceph-mon[33158]: pgmap v1237: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:07.090 INFO:tasks.workunit.client.0.smithi102.stderr:++ seq 1 10 2024-09-11T08:19:07.091 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 10` 2024-09-11T08:19:07.091 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 06b01559-be24-4d8f-9511-86207d32787f put obj1 /etc/passwd 2024-09-11T08:19:07.124 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 10` 2024-09-11T08:19:07.124 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 06b01559-be24-4d8f-9511-86207d32787f put obj2 /etc/passwd 2024-09-11T08:19:07.156 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 10` 2024-09-11T08:19:07.156 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 06b01559-be24-4d8f-9511-86207d32787f put obj3 /etc/passwd 2024-09-11T08:19:07.187 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 10` 2024-09-11T08:19:07.187 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 06b01559-be24-4d8f-9511-86207d32787f put obj4 /etc/passwd 2024-09-11T08:19:07.215 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 10` 2024-09-11T08:19:07.216 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 06b01559-be24-4d8f-9511-86207d32787f put obj5 /etc/passwd 2024-09-11T08:19:07.245 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 10` 2024-09-11T08:19:07.245 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 06b01559-be24-4d8f-9511-86207d32787f put obj6 /etc/passwd 2024-09-11T08:19:07.273 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 10` 2024-09-11T08:19:07.273 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 06b01559-be24-4d8f-9511-86207d32787f put obj7 /etc/passwd 2024-09-11T08:19:07.301 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 10` 2024-09-11T08:19:07.301 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 06b01559-be24-4d8f-9511-86207d32787f put obj8 /etc/passwd 2024-09-11T08:19:07.330 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 10` 2024-09-11T08:19:07.330 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 06b01559-be24-4d8f-9511-86207d32787f put obj9 /etc/passwd 2024-09-11T08:19:07.358 INFO:tasks.workunit.client.0.smithi102.stderr:+ for f in `seq 1 10` 2024-09-11T08:19:07.358 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 06b01559-be24-4d8f-9511-86207d32787f put obj10 /etc/passwd 2024-09-11T08:19:07.386 INFO:tasks.workunit.client.0.smithi102.stderr:+ sleep 30 2024-09-11T08:19:08.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:07 smithi102 ceph-mon[29849]: pgmap v1238: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:19:08.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:07 smithi102 ceph-mon[41371]: pgmap v1238: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:19:08.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:07 smithi146 ceph-mon[33158]: pgmap v1238: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:19:10.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:09 smithi102 ceph-mon[29849]: pgmap v1239: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:10.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:19:09 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:19:09] "GET /metrics HTTP/1.1" 200 228683 "" "Prometheus/2.43.0" 2024-09-11T08:19:10.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:09 smithi102 ceph-mon[41371]: pgmap v1239: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:10.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:09 smithi146 ceph-mon[33158]: pgmap v1239: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:10.683 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:19:10 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:19:12.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:11 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:19:12.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:11 smithi102 ceph-mon[41371]: pgmap v1240: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-11T08:19:12.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:11 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:19:12.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:11 smithi102 ceph-mon[29849]: pgmap v1240: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-11T08:19:12.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:11 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:19:12.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:11 smithi146 ceph-mon[33158]: pgmap v1240: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-11T08:19:13.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:12 smithi102 ceph-mon[41371]: pool '06b01559-be24-4d8f-9511-86207d32787f' is full (reached quota's max_objects: 10) 2024-09-11T08:19:13.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:12 smithi102 ceph-mon[41371]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:19:13.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:12 smithi102 ceph-mon[41371]: osdmap e761: 8 total, 8 up, 8 in 2024-09-11T08:19:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:12 smithi102 ceph-mon[29849]: pool '06b01559-be24-4d8f-9511-86207d32787f' is full (reached quota's max_objects: 10) 2024-09-11T08:19:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:12 smithi102 ceph-mon[29849]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:19:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:12 smithi102 ceph-mon[29849]: osdmap e761: 8 total, 8 up, 8 in 2024-09-11T08:19:13.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:12 smithi146 ceph-mon[33158]: pool '06b01559-be24-4d8f-9511-86207d32787f' is full (reached quota's max_objects: 10) 2024-09-11T08:19:13.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:12 smithi146 ceph-mon[33158]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-11T08:19:13.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:12 smithi146 ceph-mon[33158]: osdmap e761: 8 total, 8 up, 8 in 2024-09-11T08:19:14.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:13 smithi102 ceph-mon[29849]: pgmap v1242: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:19:14.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:13 smithi102 ceph-mon[41371]: pgmap v1242: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:19:14.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:13 smithi146 ceph-mon[33158]: pgmap v1242: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:19:14.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:19:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:19:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:19:15.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:15 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:19:15.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:15 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:19:15.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:15 smithi146 ceph-mon[33158]: pgmap v1243: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:19:15.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:15 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:19:15.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:15 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:19:15.895 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:15 smithi102 ceph-mon[29849]: pgmap v1243: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:19:15.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:15 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:19:15.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:15 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:19:15.896 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:15 smithi102 ceph-mon[41371]: pgmap v1243: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:19:18.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:17 smithi102 ceph-mon[29849]: pgmap v1244: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:19:18.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:17 smithi102 ceph-mon[41371]: pgmap v1244: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:19:18.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:17 smithi146 ceph-mon[33158]: pgmap v1244: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:19:20.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:19 smithi102 ceph-mon[29849]: pgmap v1245: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:19:20.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:19 smithi102 ceph-mon[41371]: pgmap v1245: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:19:20.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:19:19 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:19:19] "GET /metrics HTTP/1.1" 200 228701 "" "Prometheus/2.43.0" 2024-09-11T08:19:20.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:19 smithi146 ceph-mon[33158]: pgmap v1245: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-11T08:19:20.683 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:19:20 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:19:22.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:21 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:19:22.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:21 smithi102 ceph-mon[29849]: pgmap v1246: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:19:22.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:21 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:19:22.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:21 smithi102 ceph-mon[41371]: pgmap v1246: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:19:22.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:21 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:19:22.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:21 smithi146 ceph-mon[33158]: pgmap v1246: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:19:24.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:23 smithi102 ceph-mon[41371]: pgmap v1247: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 964 B/s rd, 0 op/s 2024-09-11T08:19:24.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:23 smithi102 ceph-mon[29849]: pgmap v1247: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 964 B/s rd, 0 op/s 2024-09-11T08:19:24.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:23 smithi146 ceph-mon[33158]: pgmap v1247: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 964 B/s rd, 0 op/s 2024-09-11T08:19:24.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:19:24 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:19:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:19:26.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:25 smithi102 ceph-mon[41371]: pgmap v1248: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:26.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:25 smithi102 ceph-mon[29849]: pgmap v1248: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:26.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:25 smithi146 ceph-mon[33158]: pgmap v1248: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:28.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:27 smithi102 ceph-mon[29849]: pgmap v1249: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:19:28.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:27 smithi102 ceph-mon[41371]: pgmap v1249: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:19:28.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:27 smithi146 ceph-mon[33158]: pgmap v1249: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:19:30.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:19:29 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:19:29] "GET /metrics HTTP/1.1" 200 228728 "" "Prometheus/2.43.0" 2024-09-11T08:19:30.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:29 smithi102 ceph-mon[41371]: pgmap v1250: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:30.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:29 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:19:30.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:29 smithi102 ceph-mon[29849]: pgmap v1250: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:30.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:29 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:19:30.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:29 smithi146 ceph-mon[33158]: pgmap v1250: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:30.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:29 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:19:30.683 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:19:30 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:19:32.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:31 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:19:32.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:31 smithi102 ceph-mon[41371]: pgmap v1251: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:19:32.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:31 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:19:32.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:31 smithi102 ceph-mon[29849]: pgmap v1251: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:19:32.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:31 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:19:32.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:31 smithi146 ceph-mon[33158]: pgmap v1251: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:19:33.395 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:33 smithi102 ceph-mon[41371]: pgmap v1252: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:33.395 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:33 smithi102 ceph-mon[29849]: pgmap v1252: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:33.433 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:33 smithi146 ceph-mon[33158]: pgmap v1252: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:34.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:19:34 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:19:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:19:36.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:35 smithi102 ceph-mon[41371]: pgmap v1253: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:36.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:35 smithi102 ceph-mon[29849]: pgmap v1253: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:36.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:35 smithi146 ceph-mon[33158]: pgmap v1253: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:37.388 INFO:tasks.workunit.client.0.smithi102.stderr:+ rados -p 91aaf327-ed21-437a-8aea-bd29873f531c put threemore /etc/passwd 2024-09-11T08:19:37.420 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph osd pool set-quota 91aaf327-ed21-437a-8aea-bd29873f531c max_bytes 0 2024-09-11T08:19:37.503 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.499+0000 7fe433ac0640 1 -- 172.21.15.102:0/3907493208 >> v1:172.21.15.102:6790/0 conn(0x7fe42c101560 legacy=0x7fe42c101930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:19:37.504 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.499+0000 7fe433ac0640 1 -- 172.21.15.102:0/3907493208 shutdown_connections 2024-09-11T08:19:37.504 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.499+0000 7fe433ac0640 1 -- 172.21.15.102:0/3907493208 >> 172.21.15.102:0/3907493208 conn(0x7fe42c0f6ff0 msgr2=0x7fe42c0f9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:19:37.504 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.499+0000 7fe433ac0640 1 -- 172.21.15.102:0/3907493208 shutdown_connections 2024-09-11T08:19:37.504 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.499+0000 7fe433ac0640 1 -- 172.21.15.102:0/3907493208 wait complete. 2024-09-11T08:19:37.504 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.499+0000 7fe433ac0640 1 Processor -- start 2024-09-11T08:19:37.504 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.499+0000 7fe433ac0640 1 -- start start 2024-09-11T08:19:37.504 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe433ac0640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe42c11bd90 con 0x7fe42c107eb0 2024-09-11T08:19:37.504 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe433ac0640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe42c11cf90 con 0x7fe42c104320 2024-09-11T08:19:37.504 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe433ac0640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe42c11e190 con 0x7fe42c101560 2024-09-11T08:19:37.505 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe43185c640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7fe42c101560 0x7fe42c0fe210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:45832/0 (socket says 172.21.15.102:45832) 2024-09-11T08:19:37.505 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe43185c640 1 -- 172.21.15.102:0/202046959 learned_addr learned my addr 172.21.15.102:0/202046959 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:19:37.505 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4115197200 0 0) 0x7fe42c11e190 con 0x7fe42c101560 2024-09-11T08:19:37.505 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4180034e0 con 0x7fe42c101560 2024-09-11T08:19:37.505 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1053163919 0 0) 0x7fe42c11bd90 con 0x7fe42c107eb0 2024-09-11T08:19:37.505 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe42c11e190 con 0x7fe42c107eb0 2024-09-11T08:19:37.505 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1629651372 0 0) 0x7fe42c11cf90 con 0x7fe42c104320 2024-09-11T08:19:37.505 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe42c11bd90 con 0x7fe42c104320 2024-09-11T08:19:37.505 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2012974427 0 0) 0x7fe4180034e0 con 0x7fe42c101560 2024-09-11T08:19:37.505 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe42c11cf90 con 0x7fe42c101560 2024-09-11T08:19:37.505 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fe428001370 con 0x7fe42c101560 2024-09-11T08:19:37.506 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 704886736 0 0) 0x7fe42c11e190 con 0x7fe42c107eb0 2024-09-11T08:19:37.506 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe4180034e0 con 0x7fe42c107eb0 2024-09-11T08:19:37.506 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fe424001920 con 0x7fe42c107eb0 2024-09-11T08:19:37.506 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1891490419 0 0) 0x7fe42c11bd90 con 0x7fe42c104320 2024-09-11T08:19:37.506 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe42c11e190 con 0x7fe42c104320 2024-09-11T08:19:37.506 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fe41c001840 con 0x7fe42c104320 2024-09-11T08:19:37.506 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2982900485 0 0) 0x7fe42c11cf90 con 0x7fe42c101560 2024-09-11T08:19:37.506 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 >> v1:172.21.15.146:6789/0 conn(0x7fe42c104320 legacy=0x7fe42c0fe920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:19:37.506 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 >> v1:172.21.15.102:6789/0 conn(0x7fe42c107eb0 legacy=0x7fe42c11a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:19:37.506 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c11f390 con 0x7fe42c101560 2024-09-11T08:19:37.506 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe433ac0640 1 -- 172.21.15.102:0/202046959 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe42c11e380 con 0x7fe42c101560 2024-09-11T08:19:37.507 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.500+0000 7fe433ac0640 1 -- 172.21.15.102:0/202046959 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe42c11e850 con 0x7fe42c101560 2024-09-11T08:19:37.507 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.501+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7fe428002150 con 0x7fe42c101560 2024-09-11T08:19:37.507 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.501+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fe4280035e0 con 0x7fe42c101560 2024-09-11T08:19:37.507 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.501+0000 7fe433ac0640 1 -- 172.21.15.102:0/202046959 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe410005010 con 0x7fe42c101560 2024-09-11T08:19:37.507 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.502+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7fe4280022c0 con 0x7fe42c101560 2024-09-11T08:19:37.507 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.502+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(761..761 src has 1..761) v4 ==== 8107+0+0 (unknown 2686684794 0 0) 0x7fe42808a8b0 con 0x7fe42c101560 2024-09-11T08:19:37.508 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.503+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=762}) v3 -- 0x7fe42c11cf90 con 0x7fe42c101560 2024-09-11T08:19:37.508 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.506+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fe428056e60 con 0x7fe42c101560 2024-09-11T08:19:37.665 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.663+0000 7fe433ac0640 1 -- 172.21.15.102:0/202046959 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fe4100052c0 con 0x7fe42c101560 2024-09-11T08:19:37.721 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.720+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 <== mon.2 v1:172.21.15.102:6790/0 10 ==== osd_map(762..762 src has 1..762) v4 ==== 620+0+0 (unknown 867679420 0 0) 0x7fe428056ba0 con 0x7fe42c101560 2024-09-11T08:19:37.722 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.720+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=763}) v3 -- 0x7fe42c11bd90 con 0x7fe42c101560 2024-09-11T08:19:37.726 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.724+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 <== mon.2 v1:172.21.15.102:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 91aaf327-ed21-437a-8aea-bd29873f531c v762) v1 ==== 217+0+0 (unknown 1885032984 0 0) 0x7fe428056800 con 0x7fe42c101560 2024-09-11T08:19:37.819 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:37.818+0000 7fe433ac0640 1 -- 172.21.15.102:0/202046959 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fe410000c40 con 0x7fe42c101560 2024-09-11T08:19:38.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:37 smithi102 ceph-mon[41371]: pgmap v1254: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:19:38.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:37 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/202046959' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:19:38.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:37 smithi102 ceph-mon[41371]: from='client.47351 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:19:38.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:37 smithi102 ceph-mon[29849]: pgmap v1254: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:19:38.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:37 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/202046959' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:19:38.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:37 smithi102 ceph-mon[29849]: from='client.47351 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:19:38.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:37 smithi146 ceph-mon[33158]: pgmap v1254: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:19:38.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:37 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/202046959' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:19:38.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:37 smithi146 ceph-mon[33158]: from='client.47351 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:19:38.739 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.737+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 <== mon.2 v1:172.21.15.102:6790/0 12 ==== osd_map(763..763 src has 1..763) v4 ==== 620+0+0 (unknown 3851468093 0 0) 0x7fe428057790 con 0x7fe42c101560 2024-09-11T08:19:38.739 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.737+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=764}) v3 -- 0x7fe41807cd50 con 0x7fe42c101560 2024-09-11T08:19:38.743 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.742+0000 7fe4227fc640 1 -- 172.21.15.102:0/202046959 <== mon.2 v1:172.21.15.102:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 91aaf327-ed21-437a-8aea-bd29873f531c v763) v1 ==== 217+0+0 (unknown 2701156310 0 0) 0x7fe4280569e0 con 0x7fe42c101560 2024-09-11T08:19:38.744 INFO:tasks.workunit.client.0.smithi102.stderr:set-quota max_bytes = 0 for pool 91aaf327-ed21-437a-8aea-bd29873f531c 2024-09-11T08:19:38.747 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.745+0000 7fe433ac0640 1 -- 172.21.15.102:0/202046959 >> v1:172.21.15.102:6800/4281557748 conn(0x7fe41806fe60 legacy=0x7fe418072310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:19:38.747 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.745+0000 7fe433ac0640 1 -- 172.21.15.102:0/202046959 >> v1:172.21.15.102:6790/0 conn(0x7fe42c101560 legacy=0x7fe42c0fe210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:19:38.747 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.745+0000 7fe433ac0640 1 -- 172.21.15.102:0/202046959 shutdown_connections 2024-09-11T08:19:38.747 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.745+0000 7fe433ac0640 1 -- 172.21.15.102:0/202046959 >> 172.21.15.102:0/202046959 conn(0x7fe42c0f6ff0 msgr2=0x7fe42c10ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:19:38.747 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.745+0000 7fe433ac0640 1 -- 172.21.15.102:0/202046959 shutdown_connections 2024-09-11T08:19:38.747 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.745+0000 7fe433ac0640 1 -- 172.21.15.102:0/202046959 wait complete. 2024-09-11T08:19:38.757 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph osd pool set-quota 91aaf327-ed21-437a-8aea-bd29873f531c max_objects 0 2024-09-11T08:19:38.841 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.836+0000 7fb349b5d640 1 -- 172.21.15.102:0/3068116223 >> v1:172.21.15.102:6790/0 conn(0x7fb3440fd570 legacy=0x7fb3440fd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:19:38.842 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.836+0000 7fb349b5d640 1 -- 172.21.15.102:0/3068116223 shutdown_connections 2024-09-11T08:19:38.842 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.836+0000 7fb349b5d640 1 -- 172.21.15.102:0/3068116223 >> 172.21.15.102:0/3068116223 conn(0x7fb3440f6fd0 msgr2=0x7fb3440f93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:19:38.842 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.836+0000 7fb349b5d640 1 -- 172.21.15.102:0/3068116223 shutdown_connections 2024-09-11T08:19:38.842 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.836+0000 7fb349b5d640 1 -- 172.21.15.102:0/3068116223 wait complete. 2024-09-11T08:19:38.842 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.837+0000 7fb349b5d640 1 Processor -- start 2024-09-11T08:19:38.842 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.837+0000 7fb349b5d640 1 -- start start 2024-09-11T08:19:38.842 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.837+0000 7fb349b5d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb34411bf60 con 0x7fb3440fd570 2024-09-11T08:19:38.843 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.837+0000 7fb349b5d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb34411d120 con 0x7fb3440fe140 2024-09-11T08:19:38.843 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.837+0000 7fb349b5d640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb34411e320 con 0x7fb344108ea0 2024-09-11T08:19:38.843 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.837+0000 7fb3437fe640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7fb3440fd570 0x7fb3441040b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:47090/0 (socket says 172.21.15.102:47090) 2024-09-11T08:19:38.843 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.837+0000 7fb3437fe640 1 -- 172.21.15.102:0/2029173302 learned_addr learned my addr 172.21.15.102:0/2029173302 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:19:38.843 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.838+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596965767 0 0) 0x7fb34411bf60 con 0x7fb3440fd570 2024-09-11T08:19:38.843 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.838+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb32c003120 con 0x7fb3440fd570 2024-09-11T08:19:38.843 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.838+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181767972 0 0) 0x7fb34411e320 con 0x7fb344108ea0 2024-09-11T08:19:38.843 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.838+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb34411bf60 con 0x7fb344108ea0 2024-09-11T08:19:38.843 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.838+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 691501950 0 0) 0x7fb34411d120 con 0x7fb3440fe140 2024-09-11T08:19:38.843 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.838+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb34411e320 con 0x7fb3440fe140 2024-09-11T08:19:38.844 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.838+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1466810503 0 0) 0x7fb32c003120 con 0x7fb3440fd570 2024-09-11T08:19:38.844 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.838+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb34411d120 con 0x7fb3440fd570 2024-09-11T08:19:38.844 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.838+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2173178336 0 0) 0x7fb34411bf60 con 0x7fb344108ea0 2024-09-11T08:19:38.844 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.838+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb32c003120 con 0x7fb344108ea0 2024-09-11T08:19:38.844 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.838+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fb334001820 con 0x7fb3440fd570 2024-09-11T08:19:38.844 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.838+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fb33c0013d0 con 0x7fb344108ea0 2024-09-11T08:19:38.844 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.838+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3834831315 0 0) 0x7fb34411e320 con 0x7fb3440fe140 2024-09-11T08:19:38.844 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.838+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb34411bf60 con 0x7fb3440fe140 2024-09-11T08:19:38.844 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.838+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fb3380018c0 con 0x7fb3440fe140 2024-09-11T08:19:38.844 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.838+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3938573251 0 0) 0x7fb34411d120 con 0x7fb3440fd570 2024-09-11T08:19:38.844 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.838+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 >> v1:172.21.15.102:6790/0 conn(0x7fb344108ea0 legacy=0x7fb34411a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:19:38.844 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.838+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 >> v1:172.21.15.146:6789/0 conn(0x7fb3440fe140 legacy=0x7fb3441047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:19:38.844 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.838+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34411f520 con 0x7fb3440fd570 2024-09-11T08:19:38.845 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.838+0000 7fb349b5d640 1 -- 172.21.15.102:0/2029173302 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34411e510 con 0x7fb3440fd570 2024-09-11T08:19:38.845 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.838+0000 7fb349b5d640 1 -- 172.21.15.102:0/2029173302 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34411e960 con 0x7fb3440fd570 2024-09-11T08:19:38.845 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.838+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7fb334002dd0 con 0x7fb3440fd570 2024-09-11T08:19:38.845 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.838+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7fb3340036c0 con 0x7fb3440fd570 2024-09-11T08:19:38.845 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.839+0000 7fb349b5d640 1 -- 172.21.15.102:0/2029173302 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb330005010 con 0x7fb3440fd570 2024-09-11T08:19:38.845 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.840+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7fb3340038a0 con 0x7fb3440fd570 2024-09-11T08:19:38.845 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.840+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(763..763 src has 1..763) v4 ==== 8107+0+0 (unknown 1913741413 0 0) 0x7fb33408a8e0 con 0x7fb3440fd570 2024-09-11T08:19:38.845 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.840+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=764}) v3 -- 0x7fb34411d120 con 0x7fb3440fd570 2024-09-11T08:19:38.846 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:38.844+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fb334056e90 con 0x7fb3440fd570 2024-09-11T08:19:39.005 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:39.004+0000 7fb349b5d640 1 -- 172.21.15.102:0/2029173302 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fb3300052c0 con 0x7fb3440fd570 2024-09-11T08:19:39.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:38 smithi102 ceph-mon[41371]: from='client.47351 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:19:39.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:38 smithi102 ceph-mon[41371]: osdmap e762: 8 total, 8 up, 8 in 2024-09-11T08:19:39.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:38 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/202046959' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:19:39.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:38 smithi102 ceph-mon[41371]: from='client.47351 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:19:39.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:38 smithi102 ceph-mon[29849]: from='client.47351 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:19:39.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:38 smithi102 ceph-mon[29849]: osdmap e762: 8 total, 8 up, 8 in 2024-09-11T08:19:39.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:38 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/202046959' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:19:39.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:38 smithi102 ceph-mon[29849]: from='client.47351 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:19:39.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:38 smithi146 ceph-mon[33158]: from='client.47351 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:19:39.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:38 smithi146 ceph-mon[33158]: osdmap e762: 8 total, 8 up, 8 in 2024-09-11T08:19:39.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:38 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/202046959' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:19:39.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:38 smithi146 ceph-mon[33158]: from='client.47351 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-11T08:19:39.740 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:39.738+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 <== mon.0 v1:172.21.15.102:6789/0 10 ==== osd_map(764..764 src has 1..764) v4 ==== 620+0+0 (unknown 3367432129 0 0) 0x7fb334056bd0 con 0x7fb3440fd570 2024-09-11T08:19:39.740 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:39.738+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=765}) v3 -- 0x7fb32c003120 con 0x7fb3440fd570 2024-09-11T08:19:39.743 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:39.742+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 <== mon.0 v1:172.21.15.102:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 91aaf327-ed21-437a-8aea-bd29873f531c v764) v1 ==== 221+0+0 (unknown 3271500 0 0) 0x7fb334056830 con 0x7fb3440fd570 2024-09-11T08:19:39.843 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:39.841+0000 7fb349b5d640 1 -- 172.21.15.102:0/2029173302 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fb330000be0 con 0x7fb3440fd570 2024-09-11T08:19:40.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:19:39 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:19:39] "GET /metrics HTTP/1.1" 200 228728 "" "Prometheus/2.43.0" 2024-09-11T08:19:40.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:39 smithi102 ceph-mon[41371]: pgmap v1256: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:19:40.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:39 smithi102 ceph-mon[41371]: from='client.47351 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:19:40.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:39 smithi102 ceph-mon[41371]: osdmap e763: 8 total, 8 up, 8 in 2024-09-11T08:19:40.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:39 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2029173302' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:19:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:39 smithi102 ceph-mon[29849]: pgmap v1256: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:19:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:39 smithi102 ceph-mon[29849]: from='client.47351 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:19:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:39 smithi102 ceph-mon[29849]: osdmap e763: 8 total, 8 up, 8 in 2024-09-11T08:19:40.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:39 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2029173302' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:19:40.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:39 smithi146 ceph-mon[33158]: pgmap v1256: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:19:40.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:39 smithi146 ceph-mon[33158]: from='client.47351 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_bytes", "val": "0"}]': finished 2024-09-11T08:19:40.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:39 smithi146 ceph-mon[33158]: osdmap e763: 8 total, 8 up, 8 in 2024-09-11T08:19:40.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:39 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2029173302' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:19:40.683 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:19:40 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:19:40.759 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:40.757+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 <== mon.0 v1:172.21.15.102:6789/0 12 ==== osd_map(765..765 src has 1..765) v4 ==== 620+0+0 (unknown 257798907 0 0) 0x7fb3340577c0 con 0x7fb3440fd570 2024-09-11T08:19:40.759 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:40.757+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=766}) v3 -- 0x7fb34411e320 con 0x7fb3440fd570 2024-09-11T08:19:40.768 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:40.767+0000 7fb32bfff640 1 -- 172.21.15.102:0/2029173302 <== mon.0 v1:172.21.15.102:6789/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 91aaf327-ed21-437a-8aea-bd29873f531c v765) v1 ==== 221+0+0 (unknown 2153855661 0 0) 0x7fb334056a10 con 0x7fb3440fd570 2024-09-11T08:19:40.769 INFO:tasks.workunit.client.0.smithi102.stderr:set-quota max_objects = 0 for pool 91aaf327-ed21-437a-8aea-bd29873f531c 2024-09-11T08:19:40.772 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:40.770+0000 7fb349b5d640 1 -- 172.21.15.102:0/2029173302 >> v1:172.21.15.102:6800/4281557748 conn(0x7fb32c06fe00 legacy=0x7fb32c0722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:19:40.773 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:40.770+0000 7fb349b5d640 1 -- 172.21.15.102:0/2029173302 >> v1:172.21.15.102:6789/0 conn(0x7fb3440fd570 legacy=0x7fb3441040b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:19:40.773 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:40.771+0000 7fb349b5d640 1 -- 172.21.15.102:0/2029173302 shutdown_connections 2024-09-11T08:19:40.773 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:40.771+0000 7fb349b5d640 1 -- 172.21.15.102:0/2029173302 >> 172.21.15.102:0/2029173302 conn(0x7fb3440f6fd0 msgr2=0x7fb344107f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:19:40.773 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:40.771+0000 7fb349b5d640 1 -- 172.21.15.102:0/2029173302 shutdown_connections 2024-09-11T08:19:40.773 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:19:40.771+0000 7fb349b5d640 1 -- 172.21.15.102:0/2029173302 wait complete. 2024-09-11T08:19:40.783 INFO:tasks.workunit.client.0.smithi102.stderr:+ sleep 30 2024-09-11T08:19:41.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2029173302' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:19:41.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:40 smithi102 ceph-mon[41371]: osdmap e764: 8 total, 8 up, 8 in 2024-09-11T08:19:41.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:40 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2029173302' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:19:41.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2029173302' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:19:41.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:40 smithi102 ceph-mon[29849]: osdmap e764: 8 total, 8 up, 8 in 2024-09-11T08:19:41.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:40 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2029173302' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:19:41.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2029173302' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:19:41.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:40 smithi146 ceph-mon[33158]: osdmap e764: 8 total, 8 up, 8 in 2024-09-11T08:19:41.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:40 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2029173302' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-11T08:19:42.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:41 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:19:42.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:41 smithi102 ceph-mon[41371]: pgmap v1259: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 1 op/s 2024-09-11T08:19:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:41 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/2029173302' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:19:42.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:41 smithi102 ceph-mon[41371]: osdmap e765: 8 total, 8 up, 8 in 2024-09-11T08:19:42.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:41 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:19:42.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:41 smithi102 ceph-mon[29849]: pgmap v1259: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 1 op/s 2024-09-11T08:19:42.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:41 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/2029173302' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:19:42.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:41 smithi102 ceph-mon[29849]: osdmap e765: 8 total, 8 up, 8 in 2024-09-11T08:19:42.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:41 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:19:42.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:41 smithi146 ceph-mon[33158]: pgmap v1259: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.2 KiB/s wr, 1 op/s 2024-09-11T08:19:42.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:41 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/2029173302' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "field": "max_objects", "val": "0"}]': finished 2024-09-11T08:19:42.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:41 smithi146 ceph-mon[33158]: osdmap e765: 8 total, 8 up, 8 in 2024-09-11T08:19:44.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:43 smithi102 ceph-mon[41371]: pgmap v1261: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.6 KiB/s wr, 1 op/s 2024-09-11T08:19:44.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:43 smithi102 ceph-mon[29849]: pgmap v1261: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.6 KiB/s wr, 1 op/s 2024-09-11T08:19:44.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:43 smithi146 ceph-mon[33158]: pgmap v1261: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.6 KiB/s wr, 1 op/s 2024-09-11T08:19:44.933 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:44 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:19:44.934 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:19:44 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:19:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:19:45.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:44 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:19:45.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:44 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:19:46.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:45 smithi102 ceph-mon[29849]: pgmap v1262: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-11T08:19:46.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:45 smithi102 ceph-mon[41371]: pgmap v1262: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-11T08:19:46.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:45 smithi146 ceph-mon[33158]: pgmap v1262: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-11T08:19:48.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:47 smithi102 ceph-mon[29849]: pgmap v1263: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-11T08:19:48.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:47 smithi102 ceph-mon[41371]: pgmap v1263: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-11T08:19:48.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:47 smithi146 ceph-mon[33158]: pgmap v1263: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-11T08:19:50.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:49 smithi102 ceph-mon[29849]: pgmap v1264: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-09-11T08:19:50.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:49 smithi102 ceph-mon[41371]: pgmap v1264: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-09-11T08:19:50.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:19:49 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:19:49] "GET /metrics HTTP/1.1" 200 228696 "" "Prometheus/2.43.0" 2024-09-11T08:19:50.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:49 smithi146 ceph-mon[33158]: pgmap v1264: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-09-11T08:19:50.683 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:19:50 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:19:52.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:51 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:19:52.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:51 smithi102 ceph-mon[29849]: pgmap v1265: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:19:52.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:51 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:19:52.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:51 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:19:52.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:51 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:19:52.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:51 smithi102 ceph-mon[29849]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:19:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:51 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:19:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:51 smithi102 ceph-mon[41371]: pgmap v1265: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:19:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:51 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:19:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:51 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:19:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:51 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:19:52.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:51 smithi102 ceph-mon[41371]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:19:52.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:51 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:19:52.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:51 smithi146 ceph-mon[33158]: pgmap v1265: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:19:52.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:51 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T08:19:52.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:51 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T08:19:52.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:51 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T08:19:52.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:51 smithi146 ceph-mon[33158]: from='mgr.24497 ' entity='mgr.y' 2024-09-11T08:19:54.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:53 smithi102 ceph-mon[29849]: pgmap v1266: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 860 B/s rd, 0 op/s 2024-09-11T08:19:54.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:53 smithi102 ceph-mon[41371]: pgmap v1266: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 860 B/s rd, 0 op/s 2024-09-11T08:19:54.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:53 smithi146 ceph-mon[33158]: pgmap v1266: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 860 B/s rd, 0 op/s 2024-09-11T08:19:54.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:19:54 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:19:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:19:56.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:55 smithi102 ceph-mon[41371]: pgmap v1267: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:56.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:55 smithi102 ceph-mon[29849]: pgmap v1267: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:56.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:55 smithi146 ceph-mon[33158]: pgmap v1267: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:19:58.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:57 smithi102 ceph-mon[29849]: pgmap v1268: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:19:58.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:57 smithi102 ceph-mon[41371]: pgmap v1268: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:19:58.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:57 smithi146 ceph-mon[33158]: pgmap v1268: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:20:00.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:59 smithi102 ceph-mon[41371]: pgmap v1269: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:20:00.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:19:59 smithi102 ceph-mon[41371]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:20:00.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:59 smithi102 ceph-mon[29849]: pgmap v1269: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:20:00.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:19:59 smithi102 ceph-mon[29849]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:20:00.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:19:59 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:19:59] "GET /metrics HTTP/1.1" 200 228717 "" "Prometheus/2.43.0" 2024-09-11T08:20:00.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:59 smithi146 ceph-mon[33158]: pgmap v1269: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:20:00.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:19:59 smithi146 ceph-mon[33158]: from='mgr.24497 v1:172.21.15.102:0/3885948003' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T08:20:00.683 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:20:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:20:01.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:00 smithi102 ceph-mon[29849]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-09-11T08:20:01.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:00 smithi102 ceph-mon[29849]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-11T08:20:01.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:00 smithi102 ceph-mon[29849]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-11T08:20:01.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:00 smithi102 ceph-mon[29849]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-11T08:20:01.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:00 smithi102 ceph-mon[29849]: [WRN] POOL_FULL: 1 pool(s) full 2024-09-11T08:20:01.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:00 smithi102 ceph-mon[29849]: pool '06b01559-be24-4d8f-9511-86207d32787f' is full (running out of quota) 2024-09-11T08:20:01.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:00 smithi102 ceph-mon[41371]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-09-11T08:20:01.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:00 smithi102 ceph-mon[41371]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-11T08:20:01.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:00 smithi102 ceph-mon[41371]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-11T08:20:01.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:00 smithi102 ceph-mon[41371]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-11T08:20:01.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:00 smithi102 ceph-mon[41371]: [WRN] POOL_FULL: 1 pool(s) full 2024-09-11T08:20:01.146 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:00 smithi102 ceph-mon[41371]: pool '06b01559-be24-4d8f-9511-86207d32787f' is full (running out of quota) 2024-09-11T08:20:01.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:00 smithi146 ceph-mon[33158]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-09-11T08:20:01.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:00 smithi146 ceph-mon[33158]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-11T08:20:01.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:00 smithi146 ceph-mon[33158]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-11T08:20:01.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:00 smithi146 ceph-mon[33158]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-11T08:20:01.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:00 smithi146 ceph-mon[33158]: [WRN] POOL_FULL: 1 pool(s) full 2024-09-11T08:20:01.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:00 smithi146 ceph-mon[33158]: pool '06b01559-be24-4d8f-9511-86207d32787f' is full (running out of quota) 2024-09-11T08:20:02.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:01 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:20:02.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:01 smithi102 ceph-mon[41371]: pgmap v1270: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:20:02.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:01 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:20:02.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:01 smithi102 ceph-mon[29849]: pgmap v1270: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:20:02.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:01 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:20:02.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:01 smithi146 ceph-mon[33158]: pgmap v1270: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:20:04.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:03 smithi102 ceph-mon[29849]: pgmap v1271: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:20:04.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:03 smithi102 ceph-mon[41371]: pgmap v1271: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:20:04.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:03 smithi146 ceph-mon[33158]: pgmap v1271: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:20:04.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:20:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:20:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:20:06.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:05 smithi102 ceph-mon[41371]: pgmap v1272: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:20:06.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:05 smithi102 ceph-mon[29849]: pgmap v1272: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:20:06.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:05 smithi146 ceph-mon[33158]: pgmap v1272: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:20:08.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:07 smithi102 ceph-mon[41371]: pgmap v1273: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:20:08.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:07 smithi102 ceph-mon[29849]: pgmap v1273: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:20:08.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:07 smithi146 ceph-mon[33158]: pgmap v1273: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:20:10.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:09 smithi102 ceph-mon[41371]: pgmap v1274: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:20:10.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:09 smithi102 ceph-mon[29849]: pgmap v1274: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:20:10.146 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:20:09 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:20:09] "GET /metrics HTTP/1.1" 200 228717 "" "Prometheus/2.43.0" 2024-09-11T08:20:10.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:09 smithi146 ceph-mon[33158]: pgmap v1274: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T08:20:10.683 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:20:10 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug there is no tcmu-runner data available 2024-09-11T08:20:10.785 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph osd pool delete 91aaf327-ed21-437a-8aea-bd29873f531c 91aaf327-ed21-437a-8aea-bd29873f531c --yes-i-really-really-mean-it 2024-09-11T08:20:10.873 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.869+0000 7f0f1879d640 1 -- 172.21.15.102:0/3762973206 >> v1:172.21.15.102:6789/0 conn(0x7f0f10101520 legacy=0x7f0f101018f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:20:10.873 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.869+0000 7f0f1879d640 1 -- 172.21.15.102:0/3762973206 shutdown_connections 2024-09-11T08:20:10.873 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.869+0000 7f0f1879d640 1 -- 172.21.15.102:0/3762973206 >> 172.21.15.102:0/3762973206 conn(0x7f0f10074ac0 msgr2=0x7f0f100f8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:20:10.873 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.869+0000 7f0f1879d640 1 -- 172.21.15.102:0/3762973206 shutdown_connections 2024-09-11T08:20:10.873 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.869+0000 7f0f1879d640 1 -- 172.21.15.102:0/3762973206 wait complete. 2024-09-11T08:20:10.873 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.870+0000 7f0f1879d640 1 Processor -- start 2024-09-11T08:20:10.873 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.870+0000 7f0f1879d640 1 -- start start 2024-09-11T08:20:10.874 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.870+0000 7f0f1879d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0f1011ad70 con 0x7f0f10101520 2024-09-11T08:20:10.874 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.870+0000 7f0f1879d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0f1011bf30 con 0x7f0f10107e40 2024-09-11T08:20:10.874 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.870+0000 7f0f1879d640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0f1011d130 con 0x7f0f10104220 2024-09-11T08:20:10.874 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.870+0000 7f0f16539640 1 --1- >> v1:172.21.15.102:6789/0 conn(0x7f0f10101520 0x7f0f100fd340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6789/0 says I am v1:172.21.15.102:42988/0 (socket says 172.21.15.102:42988) 2024-09-11T08:20:10.874 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.870+0000 7f0f16539640 1 -- 172.21.15.102:0/3668945429 learned_addr learned my addr 172.21.15.102:0/3668945429 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:20:10.874 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.870+0000 7f0f15d38640 1 --1- 172.21.15.102:0/3668945429 >> v1:172.21.15.102:6790/0 conn(0x7f0f10104220 0x7f0f100fda50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:54784/0 (socket says 172.21.15.102:54784) 2024-09-11T08:20:10.874 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.870+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3095560479 0 0) 0x7f0f1011ad70 con 0x7f0f10101520 2024-09-11T08:20:10.874 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.870+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0efc0034e0 con 0x7f0f10101520 2024-09-11T08:20:10.874 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.870+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1656839972 0 0) 0x7f0f1011d130 con 0x7f0f10104220 2024-09-11T08:20:10.874 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.870+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f1011ad70 con 0x7f0f10104220 2024-09-11T08:20:10.874 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.870+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455176178 0 0) 0x7f0f1011bf30 con 0x7f0f10107e40 2024-09-11T08:20:10.875 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.870+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f1011d130 con 0x7f0f10107e40 2024-09-11T08:20:10.875 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.871+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4178311462 0 0) 0x7f0f1011ad70 con 0x7f0f10104220 2024-09-11T08:20:10.875 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.871+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0f1011bf30 con 0x7f0f10104220 2024-09-11T08:20:10.875 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.871+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f0f040018c0 con 0x7f0f10104220 2024-09-11T08:20:10.875 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.871+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 653491760 0 0) 0x7f0efc0034e0 con 0x7f0f10101520 2024-09-11T08:20:10.875 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.871+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0f1011ad70 con 0x7f0f10101520 2024-09-11T08:20:10.875 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.871+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f0f0c0029d0 con 0x7f0f10101520 2024-09-11T08:20:10.875 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.871+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3748114226 0 0) 0x7f0f1011d130 con 0x7f0f10107e40 2024-09-11T08:20:10.875 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.871+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0efc0034e0 con 0x7f0f10107e40 2024-09-11T08:20:10.875 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.871+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f0f08001d00 con 0x7f0f10107e40 2024-09-11T08:20:10.875 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.871+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 <== mon.2 v1:172.21.15.102:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3778903194 0 0) 0x7f0f1011bf30 con 0x7f0f10104220 2024-09-11T08:20:10.875 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.871+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 >> v1:172.21.15.146:6789/0 conn(0x7f0f10107e40 legacy=0x7f0f10119470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:20:10.875 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.871+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 >> v1:172.21.15.102:6789/0 conn(0x7f0f10101520 legacy=0x7f0f100fd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:20:10.875 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.871+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 --> v1:172.21.15.102:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1011e330 con 0x7f0f10104220 2024-09-11T08:20:10.876 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.871+0000 7f0f1879d640 1 -- 172.21.15.102:0/3668945429 --> v1:172.21.15.102:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f1011d320 con 0x7f0f10104220 2024-09-11T08:20:10.876 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.871+0000 7f0f1879d640 1 -- 172.21.15.102:0/3668945429 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f1011d7f0 con 0x7f0f10104220 2024-09-11T08:20:10.876 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.871+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 <== mon.2 v1:172.21.15.102:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f0f04002c80 con 0x7f0f10104220 2024-09-11T08:20:10.876 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.871+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 <== mon.2 v1:172.21.15.102:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f0f040059a0 con 0x7f0f10104220 2024-09-11T08:20:10.876 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.872+0000 7f0f1879d640 1 -- 172.21.15.102:0/3668945429 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef4005040 con 0x7f0f10104220 2024-09-11T08:20:10.876 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.873+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 <== mon.2 v1:172.21.15.102:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7f0f04001de0 con 0x7f0f10104220 2024-09-11T08:20:10.877 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.874+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 <== mon.2 v1:172.21.15.102:6790/0 8 ==== osd_map(765..765 src has 1..765) v4 ==== 8107+0+0 (unknown 27927733 0 0) 0x7f0f0408a520 con 0x7f0f10104220 2024-09-11T08:20:10.877 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.874+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=766}) v3 -- 0x7f0f1011bf30 con 0x7f0f10104220 2024-09-11T08:20:10.878 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:10.878+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 <== mon.2 v1:172.21.15.102:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f0f04056a70 con 0x7f0f10104220 2024-09-11T08:20:11.036 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:11.036+0000 7f0f1879d640 1 -- 172.21.15.102:0/3668945429 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd pool delete", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pool2": "91aaf327-ed21-437a-8aea-bd29873f531c", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0ef4005870 con 0x7f0f10104220 2024-09-11T08:20:11.814 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:11.813+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 <== mon.2 v1:172.21.15.102:6790/0 10 ==== osd_map(766..766 src has 1..766) v4 ==== 288+0+0 (unknown 433448212 0 0) 0x7f0f040567b0 con 0x7f0f10104220 2024-09-11T08:20:11.814 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:11.813+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 --> v1:172.21.15.102:6790/0 -- mon_subscribe({osdmap=767}) v3 -- 0x7f0f1011ad70 con 0x7f0f10104220 2024-09-11T08:20:11.819 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:11.819+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 <== mon.2 v1:172.21.15.102:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pool2": "91aaf327-ed21-437a-8aea-bd29873f531c", "yes_i_really_really_mean_it": true}]=0 pool '91aaf327-ed21-437a-8aea-bd29873f531c' removed v766) v1 ==== 248+0+0 (unknown 2096519316 0 0) 0x7f0f04056410 con 0x7f0f10104220 2024-09-11T08:20:11.914 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:11.913+0000 7f0f1879d640 1 -- 172.21.15.102:0/3668945429 --> v1:172.21.15.102:6790/0 -- mon_command({"prefix": "osd pool delete", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pool2": "91aaf327-ed21-437a-8aea-bd29873f531c", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0ef4005be0 con 0x7f0f10104220 2024-09-11T08:20:11.915 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:11.914+0000 7f0f02ffd640 1 -- 172.21.15.102:0/3668945429 <== mon.2 v1:172.21.15.102:6790/0 12 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pool2": "91aaf327-ed21-437a-8aea-bd29873f531c", "yes_i_really_really_mean_it": true}]=0 pool '91aaf327-ed21-437a-8aea-bd29873f531c' does not exist v766) v1 ==== 255+0+0 (unknown 2819059905 0 0) 0x7f0f04056410 con 0x7f0f10104220 2024-09-11T08:20:11.915 INFO:tasks.workunit.client.0.smithi102.stderr:pool '91aaf327-ed21-437a-8aea-bd29873f531c' does not exist 2024-09-11T08:20:11.917 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:11.916+0000 7f0f1879d640 1 -- 172.21.15.102:0/3668945429 >> v1:172.21.15.102:6800/4281557748 conn(0x7f0efc06fd30 legacy=0x7f0efc0721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:20:11.917 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:11.916+0000 7f0f1879d640 1 -- 172.21.15.102:0/3668945429 >> v1:172.21.15.102:6790/0 conn(0x7f0f10104220 legacy=0x7f0f100fda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:20:11.917 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:11.916+0000 7f0f1879d640 1 -- 172.21.15.102:0/3668945429 shutdown_connections 2024-09-11T08:20:11.917 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:11.916+0000 7f0f1879d640 1 -- 172.21.15.102:0/3668945429 >> 172.21.15.102:0/3668945429 conn(0x7f0f10074ac0 msgr2=0x7f0f100fbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:20:11.917 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:11.916+0000 7f0f1879d640 1 -- 172.21.15.102:0/3668945429 shutdown_connections 2024-09-11T08:20:11.917 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:11.916+0000 7f0f1879d640 1 -- 172.21.15.102:0/3668945429 wait complete. 2024-09-11T08:20:11.925 INFO:tasks.workunit.client.0.smithi102.stderr:+ ceph osd pool delete 06b01559-be24-4d8f-9511-86207d32787f 06b01559-be24-4d8f-9511-86207d32787f --yes-i-really-really-mean-it 2024-09-11T08:20:12.005 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.001+0000 7f1195b5d640 1 -- 172.21.15.102:0/1212448476 >> v1:172.21.15.102:6789/0 conn(0x7f1190102650 legacy=0x7f1190102a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:20:12.005 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.001+0000 7f1195b5d640 1 -- 172.21.15.102:0/1212448476 shutdown_connections 2024-09-11T08:20:12.005 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.001+0000 7f1195b5d640 1 -- 172.21.15.102:0/1212448476 >> 172.21.15.102:0/1212448476 conn(0x7f1190074ac0 msgr2=0x7f11900f8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:20:12.005 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.001+0000 7f1195b5d640 1 -- 172.21.15.102:0/1212448476 shutdown_connections 2024-09-11T08:20:12.005 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.001+0000 7f1195b5d640 1 -- 172.21.15.102:0/1212448476 wait complete. 2024-09-11T08:20:12.005 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.002+0000 7f1195b5d640 1 Processor -- start 2024-09-11T08:20:12.005 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.002+0000 7f1195b5d640 1 -- start start 2024-09-11T08:20:12.005 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.002+0000 7f1195b5d640 1 -- --> v1:172.21.15.102:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f119011e080 con 0x7f1190108f40 2024-09-11T08:20:12.005 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.002+0000 7f1195b5d640 1 -- --> v1:172.21.15.146:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f119011f220 con 0x7f1190105380 2024-09-11T08:20:12.006 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.002+0000 7f1195b5d640 1 -- --> v1:172.21.15.102:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f11901203c0 con 0x7f1190102650 2024-09-11T08:20:12.006 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.002+0000 7f118f7fe640 1 --1- >> v1:172.21.15.102:6790/0 conn(0x7f1190102650 0x7f11900fbc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.102:6790/0 says I am v1:172.21.15.102:54790/0 (socket says 172.21.15.102:54790) 2024-09-11T08:20:12.006 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.002+0000 7f118f7fe640 1 -- 172.21.15.102:0/3954905467 learned_addr learned my addr 172.21.15.102:0/3954905467 (peer_addr_for_me v1:172.21.15.102:0/0) 2024-09-11T08:20:12.006 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.002+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 <== mon.0 v1:172.21.15.102:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2225718956 0 0) 0x7f119011e080 con 0x7f1190108f40 2024-09-11T08:20:12.006 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.002+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 --> v1:172.21.15.102:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f117c0034e0 con 0x7f1190108f40 2024-09-11T08:20:12.006 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.002+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 <== mon.2 v1:172.21.15.102:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 615806449 0 0) 0x7f11901203c0 con 0x7f1190102650 2024-09-11T08:20:12.006 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.002+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 --> v1:172.21.15.102:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f119011e080 con 0x7f1190102650 2024-09-11T08:20:12.006 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.002+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 <== mon.1 v1:172.21.15.146:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1208981016 0 0) 0x7f119011f220 con 0x7f1190105380 2024-09-11T08:20:12.006 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.002+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 --> v1:172.21.15.146:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11901203c0 con 0x7f1190105380 2024-09-11T08:20:12.006 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.003+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 <== mon.0 v1:172.21.15.102:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3398495633 0 0) 0x7f117c0034e0 con 0x7f1190108f40 2024-09-11T08:20:12.006 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.003+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 --> v1:172.21.15.102:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f119011f220 con 0x7f1190108f40 2024-09-11T08:20:12.006 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.003+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 <== mon.0 v1:172.21.15.102:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f1188001cf0 con 0x7f1190108f40 2024-09-11T08:20:12.006 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.003+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 <== mon.2 v1:172.21.15.102:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1316416339 0 0) 0x7f119011e080 con 0x7f1190102650 2024-09-11T08:20:12.007 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.003+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 --> v1:172.21.15.102:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f117c0034e0 con 0x7f1190102650 2024-09-11T08:20:12.007 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.003+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 <== mon.2 v1:172.21.15.102:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f1180002020 con 0x7f1190102650 2024-09-11T08:20:12.007 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.003+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 <== mon.1 v1:172.21.15.146:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1417575810 0 0) 0x7f11901203c0 con 0x7f1190105380 2024-09-11T08:20:12.007 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.003+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 --> v1:172.21.15.146:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f119011e080 con 0x7f1190105380 2024-09-11T08:20:12.007 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.003+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 <== mon.1 v1:172.21.15.146:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f1178001840 con 0x7f1190105380 2024-09-11T08:20:12.007 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.003+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 <== mon.0 v1:172.21.15.102:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 617229541 0 0) 0x7f119011f220 con 0x7f1190108f40 2024-09-11T08:20:12.007 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.003+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 >> v1:172.21.15.102:6790/0 conn(0x7f1190102650 legacy=0x7f11900fbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:20:12.007 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.003+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 >> v1:172.21.15.146:6789/0 conn(0x7f1190105380 legacy=0x7f11900fc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:20:12.007 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.003+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 --> v1:172.21.15.102:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1190121560 con 0x7f1190108f40 2024-09-11T08:20:12.007 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.003+0000 7f1195b5d640 1 -- 172.21.15.102:0/3954905467 --> v1:172.21.15.102:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11901205b0 con 0x7f1190108f40 2024-09-11T08:20:12.007 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.003+0000 7f1195b5d640 1 -- 172.21.15.102:0/3954905467 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1190120a00 con 0x7f1190108f40 2024-09-11T08:20:12.007 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.003+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 <== mon.0 v1:172.21.15.102:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 3499950121 0 0) 0x7f1188002610 con 0x7f1190108f40 2024-09-11T08:20:12.007 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.003+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 <== mon.0 v1:172.21.15.102:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3327336042 0 0) 0x7f1188003570 con 0x7f1190108f40 2024-09-11T08:20:12.008 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.004+0000 7f1195b5d640 1 -- 172.21.15.102:0/3954905467 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1170005010 con 0x7f1190108f40 2024-09-11T08:20:12.008 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.004+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 <== mon.0 v1:172.21.15.102:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 1776268174 0 0) 0x7f11880021c0 con 0x7f1190108f40 2024-09-11T08:20:12.008 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.005+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 <== mon.0 v1:172.21.15.102:6789/0 8 ==== osd_map(766..766 src has 1..766) v4 ==== 7719+0+0 (unknown 2967861715 0 0) 0x7f1188087da0 con 0x7f1190108f40 2024-09-11T08:20:12.008 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.005+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=767}) v3 -- 0x7f119011f220 con 0x7f1190108f40 2024-09-11T08:20:12.009 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.009+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 <== mon.0 v1:172.21.15.102:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f1188054480 con 0x7f1190108f40 2024-09-11T08:20:12.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:11 smithi102 ceph-mon[41371]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:20:12.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:11 smithi102 ceph-mon[41371]: pgmap v1275: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:20:12.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:11 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3668945429' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pool2": "91aaf327-ed21-437a-8aea-bd29873f531c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:20:12.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:11 smithi102 ceph-mon[41371]: from='client.47363 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pool2": "91aaf327-ed21-437a-8aea-bd29873f531c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:20:12.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:11 smithi102 ceph-mon[29849]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:20:12.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:11 smithi102 ceph-mon[29849]: pgmap v1275: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:20:12.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:11 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3668945429' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pool2": "91aaf327-ed21-437a-8aea-bd29873f531c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:20:12.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:11 smithi102 ceph-mon[29849]: from='client.47363 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pool2": "91aaf327-ed21-437a-8aea-bd29873f531c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:20:12.166 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.166+0000 7f1195b5d640 1 -- 172.21.15.102:0/3954905467 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pool2": "06b01559-be24-4d8f-9511-86207d32787f", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1170005840 con 0x7f1190108f40 2024-09-11T08:20:12.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:11 smithi146 ceph-mon[33158]: from='client.24457 v1:172.21.15.146:0/1631727336' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T08:20:12.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:11 smithi146 ceph-mon[33158]: pgmap v1275: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T08:20:12.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:11 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3668945429' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pool2": "91aaf327-ed21-437a-8aea-bd29873f531c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:20:12.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:11 smithi146 ceph-mon[33158]: from='client.47363 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pool2": "91aaf327-ed21-437a-8aea-bd29873f531c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:20:12.818 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.817+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 <== mon.0 v1:172.21.15.102:6789/0 10 ==== osd_map(767..767 src has 1..767) v4 ==== 288+0+0 (unknown 3656216917 0 0) 0x7f11880b62a0 con 0x7f1190108f40 2024-09-11T08:20:12.818 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.817+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 --> v1:172.21.15.102:6789/0 -- mon_subscribe({osdmap=768}) v3 -- 0x7f11901203c0 con 0x7f1190108f40 2024-09-11T08:20:12.822 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.822+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 <== mon.0 v1:172.21.15.102:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pool2": "06b01559-be24-4d8f-9511-86207d32787f", "yes_i_really_really_mean_it": true}]=0 pool '06b01559-be24-4d8f-9511-86207d32787f' removed v767) v1 ==== 248+0+0 (unknown 740518508 0 0) 0x7f1188053e20 con 0x7f1190108f40 2024-09-11T08:20:12.923 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.923+0000 7f1195b5d640 1 -- 172.21.15.102:0/3954905467 --> v1:172.21.15.102:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pool2": "06b01559-be24-4d8f-9511-86207d32787f", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1170005330 con 0x7f1190108f40 2024-09-11T08:20:12.923 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.923+0000 7f118cff9640 1 -- 172.21.15.102:0/3954905467 <== mon.0 v1:172.21.15.102:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pool2": "06b01559-be24-4d8f-9511-86207d32787f", "yes_i_really_really_mean_it": true}]=0 pool '06b01559-be24-4d8f-9511-86207d32787f' does not exist v767) v1 ==== 255+0+0 (unknown 2013223037 0 0) 0x7f1188054000 con 0x7f1190108f40 2024-09-11T08:20:12.924 INFO:tasks.workunit.client.0.smithi102.stderr:pool '06b01559-be24-4d8f-9511-86207d32787f' does not exist 2024-09-11T08:20:12.925 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.925+0000 7f1195b5d640 1 -- 172.21.15.102:0/3954905467 >> v1:172.21.15.102:6800/4281557748 conn(0x7f117c06fee0 legacy=0x7f117c072390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:20:12.925 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.925+0000 7f1195b5d640 1 -- 172.21.15.102:0/3954905467 >> v1:172.21.15.102:6789/0 conn(0x7f1190108f40 legacy=0x7f119011c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T08:20:12.925 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.925+0000 7f1195b5d640 1 -- 172.21.15.102:0/3954905467 shutdown_connections 2024-09-11T08:20:12.926 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.925+0000 7f1195b5d640 1 -- 172.21.15.102:0/3954905467 >> 172.21.15.102:0/3954905467 conn(0x7f1190074ac0 msgr2=0x7f11900faa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T08:20:12.926 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.925+0000 7f1195b5d640 1 -- 172.21.15.102:0/3954905467 shutdown_connections 2024-09-11T08:20:12.926 INFO:tasks.workunit.client.0.smithi102.stderr:2024-09-11T08:20:12.925+0000 7f1195b5d640 1 -- 172.21.15.102:0/3954905467 wait complete. 2024-09-11T08:20:12.934 INFO:tasks.workunit.client.0.smithi102.stdout:OK 2024-09-11T08:20:12.935 INFO:tasks.workunit.client.0.smithi102.stderr:+ echo OK 2024-09-11T08:20:12.936 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T08:20:12.936 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T08:20:13.002 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-09-11T08:20:13.003 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-11T08:20:13.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:12 smithi102 ceph-mon[41371]: from='client.47363 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pool2": "91aaf327-ed21-437a-8aea-bd29873f531c", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:20:13.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:12 smithi102 ceph-mon[41371]: osdmap e766: 8 total, 8 up, 8 in 2024-09-11T08:20:13.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3668945429' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pool2": "91aaf327-ed21-437a-8aea-bd29873f531c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:20:13.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:12 smithi102 ceph-mon[41371]: from='client.47363 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pool2": "91aaf327-ed21-437a-8aea-bd29873f531c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:20:13.145 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:12 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3954905467' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pool2": "06b01559-be24-4d8f-9511-86207d32787f", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:20:13.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:12 smithi102 ceph-mon[29849]: from='client.47363 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pool2": "91aaf327-ed21-437a-8aea-bd29873f531c", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:20:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:12 smithi102 ceph-mon[29849]: osdmap e766: 8 total, 8 up, 8 in 2024-09-11T08:20:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3668945429' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pool2": "91aaf327-ed21-437a-8aea-bd29873f531c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:20:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:12 smithi102 ceph-mon[29849]: from='client.47363 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pool2": "91aaf327-ed21-437a-8aea-bd29873f531c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:20:13.146 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:12 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3954905467' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pool2": "06b01559-be24-4d8f-9511-86207d32787f", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:20:13.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:12 smithi146 ceph-mon[33158]: from='client.47363 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pool2": "91aaf327-ed21-437a-8aea-bd29873f531c", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:20:13.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:12 smithi146 ceph-mon[33158]: osdmap e766: 8 total, 8 up, 8 in 2024-09-11T08:20:13.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3668945429' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pool2": "91aaf327-ed21-437a-8aea-bd29873f531c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:20:13.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:12 smithi146 ceph-mon[33158]: from='client.47363 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "91aaf327-ed21-437a-8aea-bd29873f531c", "pool2": "91aaf327-ed21-437a-8aea-bd29873f531c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:20:13.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:12 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3954905467' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pool2": "06b01559-be24-4d8f-9511-86207d32787f", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:20:13.304 DEBUG:teuthology.parallel:result is None 2024-09-11T08:20:13.305 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T08:20:13.330 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T08:20:13.330 DEBUG:teuthology.orchestra.run.smithi102:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-11T08:20:13.395 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T08:20:13.395 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-11T08:20:13.405 INFO:tasks.cephadm:Teardown begin 2024-09-11T08:20:13.405 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:20:13.460 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:20:13.495 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-11T08:20:13.495 DEBUG:teuthology.orchestra.run.smithi102:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T08:20:13.515 DEBUG:teuthology.orchestra.run.smithi146:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T08:20:13.552 INFO:tasks.cephadm:Stopping all daemons... 2024-09-11T08:20:13.553 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-11T08:20:13.553 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-2051892c-7013-11ef-bcea-c7b262605968@mon.a 2024-09-11T08:20:13.855 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:13 smithi102 ceph-mon[41371]: pgmap v1277: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:20:13.855 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:13 smithi102 ceph-mon[41371]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:20:13.856 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3954905467' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pool2": "06b01559-be24-4d8f-9511-86207d32787f", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:20:13.856 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:13 smithi102 ceph-mon[41371]: osdmap e767: 8 total, 8 up, 8 in 2024-09-11T08:20:13.856 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:13 smithi102 ceph-mon[41371]: from='client.? v1:172.21.15.102:0/3954905467' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pool2": "06b01559-be24-4d8f-9511-86207d32787f", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:20:13.856 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:13 smithi102 systemd[1]: Stopping Ceph mon.a for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:20:13.856 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:13 smithi102 ceph-mon[29849]: pgmap v1277: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:20:13.856 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:13 smithi102 ceph-mon[29849]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:20:13.856 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3954905467' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pool2": "06b01559-be24-4d8f-9511-86207d32787f", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:20:13.856 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:13 smithi102 ceph-mon[29849]: osdmap e767: 8 total, 8 up, 8 in 2024-09-11T08:20:13.856 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:13 smithi102 ceph-mon[29849]: from='client.? v1:172.21.15.102:0/3954905467' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pool2": "06b01559-be24-4d8f-9511-86207d32787f", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:20:14.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:13 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-a[29826]: 2024-09-11T08:20:13.854+0000 7f9fcb587640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:20:14.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:13 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-a[29826]: 2024-09-11T08:20:13.854+0000 7f9fcb587640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-11T08:20:14.145 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:13 smithi102 podman[150471]: 2024-09-11 08:20:13.89627044 +0000 UTC m=+0.159821617 container died 36c0f168f3ce4234c5bf7d804f02edd9522617f37391a434b8e119cf6dc4fa24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-11T08:20:14.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:13 smithi146 ceph-mon[33158]: pgmap v1277: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T08:20:14.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:13 smithi146 ceph-mon[33158]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-11T08:20:14.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3954905467' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pool2": "06b01559-be24-4d8f-9511-86207d32787f", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T08:20:14.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:13 smithi146 ceph-mon[33158]: osdmap e767: 8 total, 8 up, 8 in 2024-09-11T08:20:14.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:13 smithi146 ceph-mon[33158]: from='client.? v1:172.21.15.102:0/3954905467' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "06b01559-be24-4d8f-9511-86207d32787f", "pool2": "06b01559-be24-4d8f-9511-86207d32787f", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T08:20:14.589 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:14 smithi102 podman[150471]: 2024-09-11 08:20:14.345490945 +0000 UTC m=+0.609042181 container cleanup 36c0f168f3ce4234c5bf7d804f02edd9522617f37391a434b8e119cf6dc4fa24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-11T08:20:14.589 INFO:journalctl@ceph.mon.a.smithi102.stdout:Sep 11 08:20:14 smithi102 bash[150471]: ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-a 2024-09-11T08:20:14.722 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@mon.a.service' 2024-09-11T08:20:14.759 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:20:14.759 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-11T08:20:14.759 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-11T08:20:14.760 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-2051892c-7013-11ef-bcea-c7b262605968@mon.c 2024-09-11T08:20:14.895 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:14 smithi102 systemd[1]: Stopping Ceph mon.c for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:20:14.933 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:20:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x[35215]: ::ffff:172.21.15.146 - - [11/Sep/2024:08:20:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T08:20:15.361 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:20:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:20:15] ENGINE Bus STOPPING 2024-09-11T08:20:15.361 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-c[41348]: 2024-09-11T08:20:15.076+0000 7fe731e47640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:20:15.362 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-c[41348]: 2024-09-11T08:20:15.076+0000 7fe731e47640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-11T08:20:15.362 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:15 smithi102 podman[150608]: 2024-09-11 08:20:15.200825327 +0000 UTC m=+0.221409423 container died 1a4e9f1b37da8a545c09af3ef44b6c0938f2428f9087e1179fbf3e8bd1777c55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-c, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:20:15.622 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:15 smithi102 podman[150608]: 2024-09-11 08:20:15.609047628 +0000 UTC m=+0.629631723 container cleanup 1a4e9f1b37da8a545c09af3ef44b6c0938f2428f9087e1179fbf3e8bd1777c55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:20:15.623 INFO:journalctl@ceph.mon.c.smithi102.stdout:Sep 11 08:20:15 smithi102 bash[150608]: ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-c 2024-09-11T08:20:15.623 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:20:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:20:15] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T08:20:15.623 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:20:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:20:15] ENGINE Bus STOPPED 2024-09-11T08:20:15.623 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:20:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:20:15] ENGINE Bus STARTING 2024-09-11T08:20:15.623 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:20:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:20:15] ENGINE Serving on http://:::9283 2024-09-11T08:20:15.623 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:20:15 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: [11/Sep/2024:08:20:15] ENGINE Bus STARTED 2024-09-11T08:20:15.939 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@mon.c.service' 2024-09-11T08:20:15.973 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:20:15.973 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-11T08:20:15.973 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-11T08:20:15.973 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-2051892c-7013-11ef-bcea-c7b262605968@mon.b 2024-09-11T08:20:16.298 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:16 smithi146 systemd[1]: Stopping Ceph mon.b for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:20:16.683 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:16 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-b[33125]: 2024-09-11T08:20:16.297+0000 7f4df68fa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:20:16.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:16 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-b[33125]: 2024-09-11T08:20:16.297+0000 7f4df68fa640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-11T08:20:16.684 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:16 smithi146 podman[72538]: 2024-09-11 08:20:16.379917756 +0000 UTC m=+0.197167640 container died f6644dfa3474d14f30c5e3a0990cb1d689d58f2e8f23e00c0c91095d47d49bd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-b, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True) 2024-09-11T08:20:17.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:16 smithi146 podman[72538]: 2024-09-11 08:20:16.979920377 +0000 UTC m=+0.797170297 container cleanup f6644dfa3474d14f30c5e3a0990cb1d689d58f2e8f23e00c0c91095d47d49bd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-11T08:20:17.183 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 11 08:20:16 smithi146 bash[72538]: ceph-2051892c-7013-11ef-bcea-c7b262605968-mon-b 2024-09-11T08:20:17.395 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@mon.b.service' 2024-09-11T08:20:17.440 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:20:17.440 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-11T08:20:17.440 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-11T08:20:17.441 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-2051892c-7013-11ef-bcea-c7b262605968@mgr.y 2024-09-11T08:20:17.840 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:20:17 smithi102 systemd[1]: Stopping Ceph mgr.y for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:20:18.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:20:17 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:20:17.839+0000 7f933ea4f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:20:18.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:20:17 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y[30100]: 2024-09-11T08:20:17.839+0000 7f933ea4f640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-11T08:20:18.145 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:20:17 smithi102 podman[150750]: 2024-09-11 08:20:17.871564681 +0000 UTC m=+0.248513154 container died 5fd918c4c3f4558db2eb245fcc2d181441b1ed6dbe0bc8bbe91d74f8f4044d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True) 2024-09-11T08:20:18.473 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:20:18 smithi102 podman[150750]: 2024-09-11 08:20:18.246564263 +0000 UTC m=+0.623512733 container cleanup 5fd918c4c3f4558db2eb245fcc2d181441b1ed6dbe0bc8bbe91d74f8f4044d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-11T08:20:18.473 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:20:18 smithi102 bash[150750]: ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y 2024-09-11T08:20:18.724 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:20:18 smithi102 podman[150769]: 2024-09-11 08:20:18.473517236 +0000 UTC m=+0.600050350 container remove 5fd918c4c3f4558db2eb245fcc2d181441b1ed6dbe0bc8bbe91d74f8f4044d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-11T08:20:18.724 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 11 08:20:18 smithi102 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@mgr.y.service: Deactivated successfully. 2024-09-11T08:20:18.727 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-11T08:20:18.761 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:20:18.761 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-11T08:20:18.761 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-11T08:20:18.761 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-2051892c-7013-11ef-bcea-c7b262605968@mgr.x 2024-09-11T08:20:19.099 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:20:18 smithi146 systemd[1]: Stopping Ceph mgr.x for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:20:19.420 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:20:19 smithi146 podman[72671]: 2024-09-11 08:20:19.09905523 +0000 UTC m=+0.142758483 container died 8b817570c442a0552c9a5f33ca183c5ac3bbc4304d770165ae4d490afc51b8c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-11T08:20:19.683 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:20:19 smithi146 podman[72671]: 2024-09-11 08:20:19.501517327 +0000 UTC m=+0.545220581 container cleanup 8b817570c442a0552c9a5f33ca183c5ac3bbc4304d770165ae4d490afc51b8c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:20:19.683 INFO:journalctl@ceph.mgr.x.smithi146.stdout:Sep 11 08:20:19 smithi146 bash[72671]: ceph-2051892c-7013-11ef-bcea-c7b262605968-mgr-x 2024-09-11T08:20:19.845 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-11T08:20:19.881 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:20:19.881 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-11T08:20:19.881 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-11T08:20:19.881 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.0 2024-09-11T08:20:20.261 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 08:20:19 smithi102 systemd[1]: Stopping Ceph osd.0 for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:20:20.261 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:20:20 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:20:20.147Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused" 2024-09-11T08:20:20.261 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:20:20 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:20:20.148Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused" 2024-09-11T08:20:20.645 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 08:20:20 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-0[45637]: 2024-09-11T08:20:20.260+0000 7f4e1ff25640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:20:20.645 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 08:20:20 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-0[45637]: 2024-09-11T08:20:20.260+0000 7f4e1ff25640 -1 osd.0 767 *** Got signal Terminated *** 2024-09-11T08:20:20.645 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 08:20:20 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-0[45637]: 2024-09-11T08:20:20.260+0000 7f4e1ff25640 -1 osd.0 767 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:20:25.895 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 08:20:25 smithi102 podman[150884]: 2024-09-11 08:20:25.624029482 +0000 UTC m=+5.562864578 container died 506f0376cbe878f629d7975b7351e5a0cb27e22a6b9caf8d20309c081110bf89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-11T08:20:26.152 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 08:20:25 smithi102 podman[150884]: 2024-09-11 08:20:25.934035303 +0000 UTC m=+5.872870398 container cleanup 506f0376cbe878f629d7975b7351e5a0cb27e22a6b9caf8d20309c081110bf89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:20:26.152 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 08:20:25 smithi102 bash[150884]: ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-0 2024-09-11T08:20:26.559 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 08:20:26 smithi102 podman[150906]: 2024-09-11 08:20:26.152343242 +0000 UTC m=+0.528450457 container remove 506f0376cbe878f629d7975b7351e5a0cb27e22a6b9caf8d20309c081110bf89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-0, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:20:26.856 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 08:20:26 smithi102 podman[150999]: 2024-09-11 08:20:26.461879736 +0000 UTC m=+0.015898241 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:20:26.856 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 08:20:26 smithi102 podman[150999]: 2024-09-11 08:20:26.561452661 +0000 UTC m=+0.115471209 container create 7c619b278a178e937e4a11b643c37eb803b103806b46eeff166cfa86f6361be3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T08:20:26.856 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 08:20:26 smithi102 podman[150999]: 2024-09-11 08:20:26.772500168 +0000 UTC m=+0.326518674 container init 7c619b278a178e937e4a11b643c37eb803b103806b46eeff166cfa86f6361be3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-11T08:20:26.856 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 08:20:26 smithi102 podman[150999]: 2024-09-11 08:20:26.777371027 +0000 UTC m=+0.331389538 container start 7c619b278a178e937e4a11b643c37eb803b103806b46eeff166cfa86f6361be3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T08:20:26.856 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 08:20:26 smithi102 podman[150999]: 2024-09-11 08:20:26.855639798 +0000 UTC m=+0.409658309 container attach 7c619b278a178e937e4a11b643c37eb803b103806b46eeff166cfa86f6361be3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:20:27.145 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 08:20:26 smithi102 podman[150999]: 2024-09-11 08:20:26.951062883 +0000 UTC m=+0.505081386 container died 7c619b278a178e937e4a11b643c37eb803b103806b46eeff166cfa86f6361be3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:20:27.486 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.0.service' 2024-09-11T08:20:27.511 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 08:20:27 smithi102 podman[151042]: 2024-09-11 08:20:27.46743127 +0000 UTC m=+0.510568438 container remove 7c619b278a178e937e4a11b643c37eb803b103806b46eeff166cfa86f6361be3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:20:27.511 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 08:20:27 smithi102 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.0.service: Deactivated successfully. 2024-09-11T08:20:27.511 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 08:20:27 smithi102 systemd[1]: Stopped Ceph osd.0 for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T08:20:27.512 INFO:journalctl@ceph.osd.0.smithi102.stdout:Sep 11 08:20:27 smithi102 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.0.service: Consumed 32.018s CPU time. 2024-09-11T08:20:27.524 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:20:27.524 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-11T08:20:27.524 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-11T08:20:27.525 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.1 2024-09-11T08:20:27.777 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 08:20:27 smithi102 systemd[1]: Stopping Ceph osd.1 for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:20:28.145 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 08:20:27 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-1[51767]: 2024-09-11T08:20:27.929+0000 7fee1157c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:20:28.145 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 08:20:27 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-1[51767]: 2024-09-11T08:20:27.929+0000 7fee1157c640 -1 osd.1 767 *** Got signal Terminated *** 2024-09-11T08:20:28.145 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 08:20:27 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-1[51767]: 2024-09-11T08:20:27.929+0000 7fee1157c640 -1 osd.1 767 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:20:30.645 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:20:30 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:20:30.147Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused" 2024-09-11T08:20:30.645 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:20:30 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:20:30.148Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused" 2024-09-11T08:20:30.645 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:20:30 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:20:30.148Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused" 2024-09-11T08:20:33.554 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 08:20:33 smithi102 podman[151128]: 2024-09-11 08:20:33.285832425 +0000 UTC m=+5.485510045 container died f6efa4d15b8847b9e6bb0de3e7c9225584568909a84f716dd07fc0b7ec87ae74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:20:33.895 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 08:20:33 smithi102 podman[151128]: 2024-09-11 08:20:33.653210748 +0000 UTC m=+5.852888358 container cleanup f6efa4d15b8847b9e6bb0de3e7c9225584568909a84f716dd07fc0b7ec87ae74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-11T08:20:33.895 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 08:20:33 smithi102 bash[151128]: ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-1 2024-09-11T08:20:34.268 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 08:20:33 smithi102 podman[151164]: 2024-09-11 08:20:33.940483023 +0000 UTC m=+0.653202834 container remove f6efa4d15b8847b9e6bb0de3e7c9225584568909a84f716dd07fc0b7ec87ae74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:20:34.583 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 08:20:34 smithi102 podman[151259]: 2024-09-11 08:20:34.171356782 +0000 UTC m=+0.015646447 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:20:34.583 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 08:20:34 smithi102 podman[151259]: 2024-09-11 08:20:34.289282656 +0000 UTC m=+0.133572301 container create ba4638edf884163ea314f9fa8944a758a65b63838f6c34ef7126d178bfaf3552 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-11T08:20:34.583 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 08:20:34 smithi102 podman[151259]: 2024-09-11 08:20:34.499956529 +0000 UTC m=+0.344246184 container init ba4638edf884163ea314f9fa8944a758a65b63838f6c34ef7126d178bfaf3552 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-11T08:20:34.583 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 08:20:34 smithi102 podman[151259]: 2024-09-11 08:20:34.505139537 +0000 UTC m=+0.349429198 container start ba4638edf884163ea314f9fa8944a758a65b63838f6c34ef7126d178bfaf3552 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-11T08:20:34.583 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 08:20:34 smithi102 podman[151259]: 2024-09-11 08:20:34.583176297 +0000 UTC m=+0.427465948 container attach ba4638edf884163ea314f9fa8944a758a65b63838f6c34ef7126d178bfaf3552 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731) 2024-09-11T08:20:34.895 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 11 08:20:34 smithi102 podman[151259]: 2024-09-11 08:20:34.682197443 +0000 UTC m=+0.526487112 container died ba4638edf884163ea314f9fa8944a758a65b63838f6c34ef7126d178bfaf3552 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-11T08:20:35.250 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.1.service' 2024-09-11T08:20:35.290 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:20:35.290 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-11T08:20:35.290 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-11T08:20:35.290 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.2 2024-09-11T08:20:35.633 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 08:20:35 smithi102 systemd[1]: Stopping Ceph osd.2 for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:20:35.895 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 08:20:35 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-2[58066]: 2024-09-11T08:20:35.633+0000 7f33f6801640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:20:35.895 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 08:20:35 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-2[58066]: 2024-09-11T08:20:35.633+0000 7f33f6801640 -1 osd.2 767 *** Got signal Terminated *** 2024-09-11T08:20:35.895 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 08:20:35 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-2[58066]: 2024-09-11T08:20:35.633+0000 7f33f6801640 -1 osd.2 767 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:20:40.645 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:20:40 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:20:40.148Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused" 2024-09-11T08:20:40.645 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:20:40 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:20:40.149Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused" 2024-09-11T08:20:40.645 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:20:40 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:20:40.150Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused" 2024-09-11T08:20:41.291 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 08:20:40 smithi102 podman[151386]: 2024-09-11 08:20:40.967270029 +0000 UTC m=+5.451518164 container died 1a3e3b58ba96b4fe82e8fb45c29317fc8d3e6257b435cb81285386167e967d22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-11T08:20:41.565 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 08:20:41 smithi102 podman[151386]: 2024-09-11 08:20:41.389559876 +0000 UTC m=+5.873808021 container cleanup 1a3e3b58ba96b4fe82e8fb45c29317fc8d3e6257b435cb81285386167e967d22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-11T08:20:41.566 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 08:20:41 smithi102 bash[151386]: ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-2 2024-09-11T08:20:41.895 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 08:20:41 smithi102 podman[151405]: 2024-09-11 08:20:41.565768052 +0000 UTC m=+0.590055890 container remove 1a3e3b58ba96b4fe82e8fb45c29317fc8d3e6257b435cb81285386167e967d22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-11T08:20:42.310 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 08:20:42 smithi102 podman[151503]: 2024-09-11 08:20:41.905582409 +0000 UTC m=+0.015580575 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:20:42.311 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 08:20:42 smithi102 podman[151503]: 2024-09-11 08:20:42.016639292 +0000 UTC m=+0.126637438 container create cf34ea012f85d51227a3c4e5188220e34c4594b93c69fe026e4b1f6ddd34ed89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:20:42.311 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 08:20:42 smithi102 podman[151503]: 2024-09-11 08:20:42.227350585 +0000 UTC m=+0.337348738 container init cf34ea012f85d51227a3c4e5188220e34c4594b93c69fe026e4b1f6ddd34ed89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:20:42.311 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 08:20:42 smithi102 podman[151503]: 2024-09-11 08:20:42.232878278 +0000 UTC m=+0.342876432 container start cf34ea012f85d51227a3c4e5188220e34c4594b93c69fe026e4b1f6ddd34ed89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-11T08:20:42.311 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 08:20:42 smithi102 podman[151503]: 2024-09-11 08:20:42.31049854 +0000 UTC m=+0.420496697 container attach cf34ea012f85d51227a3c4e5188220e34c4594b93c69fe026e4b1f6ddd34ed89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-11T08:20:42.645 INFO:journalctl@ceph.osd.2.smithi102.stdout:Sep 11 08:20:42 smithi102 podman[151503]: 2024-09-11 08:20:42.408260197 +0000 UTC m=+0.518258363 container died cf34ea012f85d51227a3c4e5188220e34c4594b93c69fe026e4b1f6ddd34ed89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0) 2024-09-11T08:20:43.039 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.2.service' 2024-09-11T08:20:43.077 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:20:43.078 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-11T08:20:43.078 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-11T08:20:43.078 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.3 2024-09-11T08:20:43.394 INFO:journalctl@ceph.osd.3.smithi102.stdout:Sep 11 08:20:43 smithi102 systemd[1]: Stopping Ceph osd.3 for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:20:43.645 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:20:43 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:20:43.574Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused" 2024-09-11T08:20:43.645 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:20:43 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:20:43.575Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused" 2024-09-11T08:20:43.645 INFO:journalctl@ceph.osd.3.smithi102.stdout:Sep 11 08:20:43 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-3[64421]: 2024-09-11T08:20:43.393+0000 7f3a04db6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:20:43.645 INFO:journalctl@ceph.osd.3.smithi102.stdout:Sep 11 08:20:43 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-3[64421]: 2024-09-11T08:20:43.393+0000 7f3a04db6640 -1 osd.3 767 *** Got signal Terminated *** 2024-09-11T08:20:43.645 INFO:journalctl@ceph.osd.3.smithi102.stdout:Sep 11 08:20:43 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-3[64421]: 2024-09-11T08:20:43.393+0000 7f3a04db6640 -1 osd.3 767 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:20:47.353 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:20:47 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:20:47.025+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:23.477782+0000 front 2024-09-11T08:20:23.478003+0000 (oldest deadline 2024-09-11T08:20:46.377657+0000) 2024-09-11T08:20:47.395 INFO:journalctl@ceph.osd.3.smithi102.stdout:Sep 11 08:20:47 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-3[64421]: 2024-09-11T08:20:47.054+0000 7f39ffbcb640 -1 osd.3 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.473736+0000 front 2024-09-11T08:20:22.473692+0000 (oldest deadline 2024-09-11T08:20:46.573849+0000) 2024-09-11T08:20:47.626 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:20:47 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:20:47.486+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:20:48.395 INFO:journalctl@ceph.osd.3.smithi102.stdout:Sep 11 08:20:48 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-3[64421]: 2024-09-11T08:20:48.068+0000 7f39ffbcb640 -1 osd.3 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.473736+0000 front 2024-09-11T08:20:22.473692+0000 (oldest deadline 2024-09-11T08:20:46.573849+0000) 2024-09-11T08:20:48.433 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:20:48 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:20:48.055+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:23.477782+0000 front 2024-09-11T08:20:23.478003+0000 (oldest deadline 2024-09-11T08:20:46.377657+0000) 2024-09-11T08:20:48.933 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:20:48 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:20:48.497+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:20:49.010 INFO:journalctl@ceph.osd.3.smithi102.stdout:Sep 11 08:20:48 smithi102 podman[151633]: 2024-09-11 08:20:48.749152279 +0000 UTC m=+5.446675699 container died 968195807ac235733a36567cf244b1c4c77b4bbaf8def4f1ea49fa1c3f7a13dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-11T08:20:49.395 INFO:journalctl@ceph.osd.3.smithi102.stdout:Sep 11 08:20:49 smithi102 podman[151633]: 2024-09-11 08:20:49.091177187 +0000 UTC m=+5.788700607 container cleanup 968195807ac235733a36567cf244b1c4c77b4bbaf8def4f1ea49fa1c3f7a13dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:20:49.395 INFO:journalctl@ceph.osd.3.smithi102.stdout:Sep 11 08:20:49 smithi102 bash[151633]: ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-3 2024-09-11T08:20:49.395 INFO:journalctl@ceph.osd.3.smithi102.stdout:Sep 11 08:20:49 smithi102 podman[151653]: 2024-09-11 08:20:49.259053805 +0000 UTC m=+0.508941246 container remove 968195807ac235733a36567cf244b1c4c77b4bbaf8def4f1ea49fa1c3f7a13dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-3, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-11T08:20:49.433 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:20:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:20:49.040+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:23.477782+0000 front 2024-09-11T08:20:23.478003+0000 (oldest deadline 2024-09-11T08:20:46.377657+0000) 2024-09-11T08:20:49.705 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:20:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:20:49.448+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:20:49.921 INFO:journalctl@ceph.osd.3.smithi102.stdout:Sep 11 08:20:49 smithi102 podman[151749]: 2024-09-11 08:20:49.651513129 +0000 UTC m=+0.111985940 container create b2580a0bc605fd7036bbfc15eec81ee1e8c0093a31aafaa521ebb4635db06991 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:20:49.921 INFO:journalctl@ceph.osd.3.smithi102.stdout:Sep 11 08:20:49 smithi102 podman[151749]: 2024-09-11 08:20:49.555291047 +0000 UTC m=+0.015763847 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:20:49.921 INFO:journalctl@ceph.osd.3.smithi102.stdout:Sep 11 08:20:49 smithi102 podman[151749]: 2024-09-11 08:20:49.837389154 +0000 UTC m=+0.297861954 container init b2580a0bc605fd7036bbfc15eec81ee1e8c0093a31aafaa521ebb4635db06991 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:20:49.921 INFO:journalctl@ceph.osd.3.smithi102.stdout:Sep 11 08:20:49 smithi102 podman[151749]: 2024-09-11 08:20:49.842547856 +0000 UTC m=+0.303020659 container start b2580a0bc605fd7036bbfc15eec81ee1e8c0093a31aafaa521ebb4635db06991 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-11T08:20:49.921 INFO:journalctl@ceph.osd.3.smithi102.stdout:Sep 11 08:20:49 smithi102 podman[151749]: 2024-09-11 08:20:49.920451331 +0000 UTC m=+0.380924144 container attach b2580a0bc605fd7036bbfc15eec81ee1e8c0093a31aafaa521ebb4635db06991 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T08:20:50.045 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:20:49 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:20:49.704+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:20:50.356 INFO:journalctl@ceph.osd.3.smithi102.stdout:Sep 11 08:20:50 smithi102 podman[151749]: 2024-09-11 08:20:50.022010825 +0000 UTC m=+0.482483637 container died b2580a0bc605fd7036bbfc15eec81ee1e8c0093a31aafaa521ebb4635db06991 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-09-11T08:20:50.356 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:20:50 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:20:50.149Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused" 2024-09-11T08:20:50.357 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:20:50 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:20:50.149Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused" 2024-09-11T08:20:50.357 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:20:50 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:20:50.150Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused" 2024-09-11T08:20:50.404 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:20:50 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:20:50.043+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:23.477782+0000 front 2024-09-11T08:20:23.478003+0000 (oldest deadline 2024-09-11T08:20:46.377657+0000) 2024-09-11T08:20:50.678 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.3.service' 2024-09-11T08:20:50.683 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 11 08:20:50 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-4[38466]: 2024-09-11T08:20:50.624+0000 7f8813a74640 -1 osd.4 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:25.038516+0000 front 2024-09-11T08:20:25.038375+0000 (oldest deadline 2024-09-11T08:20:49.738149+0000) 2024-09-11T08:20:50.683 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:20:50 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:20:50.403+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:20:50.717 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:20:50.718 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-11T08:20:50.718 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-11T08:20:50.718 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.4 2024-09-11T08:20:51.013 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:20:50 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:20:50.711+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:20:51.013 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 11 08:20:50 smithi146 systemd[1]: Stopping Ceph osd.4 for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:20:51.014 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 11 08:20:51 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-4[38466]: 2024-09-11T08:20:51.011+0000 7f881a462640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:20:51.433 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 11 08:20:51 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-4[38466]: 2024-09-11T08:20:51.011+0000 7f881a462640 -1 osd.4 767 *** Got signal Terminated *** 2024-09-11T08:20:51.433 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 11 08:20:51 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-4[38466]: 2024-09-11T08:20:51.011+0000 7f881a462640 -1 osd.4 767 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:20:51.434 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:20:51 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:20:51.056+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:23.477782+0000 front 2024-09-11T08:20:23.478003+0000 (oldest deadline 2024-09-11T08:20:46.377657+0000) 2024-09-11T08:20:51.750 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 11 08:20:51 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-4[38466]: 2024-09-11T08:20:51.642+0000 7f8813a74640 -1 osd.4 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:25.038516+0000 front 2024-09-11T08:20:25.038375+0000 (oldest deadline 2024-09-11T08:20:49.738149+0000) 2024-09-11T08:20:51.750 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:20:51 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:20:51.433+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:20:52.048 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:20:51 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:20:51.748+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:20:52.433 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:20:52 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:20:52.047+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:23.477782+0000 front 2024-09-11T08:20:23.478003+0000 (oldest deadline 2024-09-11T08:20:46.377657+0000) 2024-09-11T08:20:52.768 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 11 08:20:52 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-4[38466]: 2024-09-11T08:20:52.654+0000 7f8813a74640 -1 osd.4 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:25.038516+0000 front 2024-09-11T08:20:25.038375+0000 (oldest deadline 2024-09-11T08:20:49.738149+0000) 2024-09-11T08:20:52.768 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:20:52 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:20:52.447+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:20:53.183 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:20:53 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:20:52.999+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:23.477782+0000 front 2024-09-11T08:20:23.478003+0000 (oldest deadline 2024-09-11T08:20:46.377657+0000) 2024-09-11T08:20:53.184 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:20:52 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:20:52.768+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:20:53.895 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:20:53 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:20:53.574Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=2 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused" 2024-09-11T08:20:53.895 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:20:53 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:20:53.575Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused" 2024-09-11T08:20:53.896 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:20:53 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:20:53.575Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused" 2024-09-11T08:20:53.933 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:20:53 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:20:53.489+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:20:53.934 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 11 08:20:53 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-4[38466]: 2024-09-11T08:20:53.665+0000 7f8813a74640 -1 osd.4 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:25.038516+0000 front 2024-09-11T08:20:25.038375+0000 (oldest deadline 2024-09-11T08:20:49.738149+0000) 2024-09-11T08:20:53.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:20:53 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:20:53.734+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:20:54.433 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:20:53 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:20:53.983+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:23.477782+0000 front 2024-09-11T08:20:23.478003+0000 (oldest deadline 2024-09-11T08:20:46.377657+0000) 2024-09-11T08:20:54.779 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 11 08:20:54 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-4[38466]: 2024-09-11T08:20:54.699+0000 7f8813a74640 -1 osd.4 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:25.038516+0000 front 2024-09-11T08:20:25.038375+0000 (oldest deadline 2024-09-11T08:20:49.738149+0000) 2024-09-11T08:20:54.779 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:20:54 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:20:54.507+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:20:55.034 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:20:54 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:20:54.777+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:20:55.433 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:20:55 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:20:55.033+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:23.477782+0000 front 2024-09-11T08:20:23.478003+0000 (oldest deadline 2024-09-11T08:20:46.377657+0000) 2024-09-11T08:20:55.433 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:20:55 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:20:55.033+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:28.678582+0000 front 2024-09-11T08:20:28.678543+0000 (oldest deadline 2024-09-11T08:20:54.578672+0000) 2024-09-11T08:20:55.750 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 11 08:20:55 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-4[38466]: 2024-09-11T08:20:55.651+0000 7f8813a74640 -1 osd.4 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:25.038516+0000 front 2024-09-11T08:20:25.038375+0000 (oldest deadline 2024-09-11T08:20:49.738149+0000) 2024-09-11T08:20:55.751 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:20:55 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:20:55.459+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:20:55.751 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:20:55 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:20:55.749+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:20:56.348 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:20:56 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:20:56.016+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:23.477782+0000 front 2024-09-11T08:20:23.478003+0000 (oldest deadline 2024-09-11T08:20:46.377657+0000) 2024-09-11T08:20:56.348 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:20:56 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:20:56.016+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:28.678582+0000 front 2024-09-11T08:20:28.678543+0000 (oldest deadline 2024-09-11T08:20:54.578672+0000) 2024-09-11T08:20:56.638 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 11 08:20:56 smithi146 podman[72822]: 2024-09-11 08:20:56.348006825 +0000 UTC m=+5.430602786 container died a0b05246c272ac29c7eb13e64b22076a1079b28115627441a465be2e283b4bfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-11T08:20:56.638 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:20:56 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:20:56.441+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:20:56.638 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:20:56 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:20:56.441+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:31.909322+0000 front 2024-09-11T08:20:31.909100+0000 (oldest deadline 2024-09-11T08:20:56.009130+0000) 2024-09-11T08:20:56.933 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 11 08:20:56 smithi146 podman[72822]: 2024-09-11 08:20:56.735631866 +0000 UTC m=+5.818227824 container cleanup a0b05246c272ac29c7eb13e64b22076a1079b28115627441a465be2e283b4bfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:20:56.933 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 11 08:20:56 smithi146 bash[72822]: ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-4 2024-09-11T08:20:56.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:20:56 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:20:56.704+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:20:56.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:20:56 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:20:56.704+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:32.604870+0000 front 2024-09-11T08:20:32.604797+0000 (oldest deadline 2024-09-11T08:20:56.104801+0000) 2024-09-11T08:20:57.345 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 11 08:20:56 smithi146 podman[72842]: 2024-09-11 08:20:56.986742029 +0000 UTC m=+0.632282832 container remove a0b05246c272ac29c7eb13e64b22076a1079b28115627441a465be2e283b4bfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-4, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-11T08:20:57.345 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:20:56 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:20:56.987+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:23.477782+0000 front 2024-09-11T08:20:23.478003+0000 (oldest deadline 2024-09-11T08:20:46.377657+0000) 2024-09-11T08:20:57.345 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:20:56 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:20:56.987+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:28.678582+0000 front 2024-09-11T08:20:28.678543+0000 (oldest deadline 2024-09-11T08:20:54.578672+0000) 2024-09-11T08:20:57.609 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:20:57 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:20:57.483+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:20:57.609 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:20:57 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:20:57.483+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:31.909322+0000 front 2024-09-11T08:20:31.909100+0000 (oldest deadline 2024-09-11T08:20:56.009130+0000) 2024-09-11T08:20:57.610 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 11 08:20:57 smithi146 podman[72938]: 2024-09-11 08:20:57.247845065 +0000 UTC m=+0.015847710 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:20:57.610 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 11 08:20:57 smithi146 podman[72938]: 2024-09-11 08:20:57.372440405 +0000 UTC m=+0.140443039 container create 99c32ea0864e7b94440b4a31bc268802552062525df99347d0b3a37a9e1b4688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-11T08:20:57.610 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 11 08:20:57 smithi146 podman[72938]: 2024-09-11 08:20:57.609683339 +0000 UTC m=+0.377685976 container init 99c32ea0864e7b94440b4a31bc268802552062525df99347d0b3a37a9e1b4688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T08:20:57.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:20:57 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:20:57.670+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:20:57.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:20:57 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:20:57.670+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:32.604870+0000 front 2024-09-11T08:20:32.604797+0000 (oldest deadline 2024-09-11T08:20:56.104801+0000) 2024-09-11T08:20:57.934 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 11 08:20:57 smithi146 podman[72938]: 2024-09-11 08:20:57.614610422 +0000 UTC m=+0.382613053 container start 99c32ea0864e7b94440b4a31bc268802552062525df99347d0b3a37a9e1b4688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:20:57.934 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 11 08:20:57 smithi146 podman[72938]: 2024-09-11 08:20:57.69846341 +0000 UTC m=+0.466466053 container attach 99c32ea0864e7b94440b4a31bc268802552062525df99347d0b3a37a9e1b4688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-11T08:20:57.934 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 11 08:20:57 smithi146 podman[72938]: 2024-09-11 08:20:57.802581287 +0000 UTC m=+0.570583919 container died 99c32ea0864e7b94440b4a31bc268802552062525df99347d0b3a37a9e1b4688 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-11T08:20:58.418 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:20:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:20:58.004+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:23.477782+0000 front 2024-09-11T08:20:23.478003+0000 (oldest deadline 2024-09-11T08:20:46.377657+0000) 2024-09-11T08:20:58.418 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:20:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:20:58.004+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:28.678582+0000 front 2024-09-11T08:20:28.678543+0000 (oldest deadline 2024-09-11T08:20:54.578672+0000) 2024-09-11T08:20:58.437 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.4.service' 2024-09-11T08:20:58.476 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:20:58.476 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-11T08:20:58.476 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-11T08:20:58.476 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.5 2024-09-11T08:20:58.683 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:20:58 smithi146 systemd[1]: Stopping Ceph osd.5 for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:20:58.684 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:20:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:20:58.482+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:20:58.684 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:20:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:20:58.482+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:31.909322+0000 front 2024-09-11T08:20:31.909100+0000 (oldest deadline 2024-09-11T08:20:56.009130+0000) 2024-09-11T08:20:59.042 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:20:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:20:58.798+0000 7fc6a25be640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:20:59.042 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:20:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:20:58.798+0000 7fc6a25be640 -1 osd.5 767 *** Got signal Terminated *** 2024-09-11T08:20:59.042 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:20:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:20:58.798+0000 7fc6a25be640 -1 osd.5 767 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:20:59.042 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:20:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:20:58.713+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:20:59.043 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:20:58 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:20:58.713+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:32.604870+0000 front 2024-09-11T08:20:32.604797+0000 (oldest deadline 2024-09-11T08:20:56.104801+0000) 2024-09-11T08:20:59.433 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:20:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:20:59.041+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:23.477782+0000 front 2024-09-11T08:20:23.478003+0000 (oldest deadline 2024-09-11T08:20:46.377657+0000) 2024-09-11T08:20:59.433 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:20:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:20:59.041+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:28.678582+0000 front 2024-09-11T08:20:28.678543+0000 (oldest deadline 2024-09-11T08:20:54.578672+0000) 2024-09-11T08:20:59.934 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:20:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:20:59.501+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:20:59.934 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:20:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:20:59.501+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:31.909322+0000 front 2024-09-11T08:20:31.909100+0000 (oldest deadline 2024-09-11T08:20:56.009130+0000) 2024-09-11T08:20:59.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:20:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:20:59.696+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:20:59.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:20:59 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:20:59.696+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:32.604870+0000 front 2024-09-11T08:20:32.604797+0000 (oldest deadline 2024-09-11T08:20:56.104801+0000) 2024-09-11T08:21:00.433 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:21:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:21:00.003+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:23.477782+0000 front 2024-09-11T08:20:23.478003+0000 (oldest deadline 2024-09-11T08:20:46.377657+0000) 2024-09-11T08:21:00.434 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:21:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:21:00.003+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:28.678582+0000 front 2024-09-11T08:20:28.678543+0000 (oldest deadline 2024-09-11T08:20:54.578672+0000) 2024-09-11T08:21:00.467 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:00 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:00.150Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused" 2024-09-11T08:21:00.467 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:00 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:00.150Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused" 2024-09-11T08:21:00.467 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:00 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:00.151Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused" 2024-09-11T08:21:00.933 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:00.485+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:21:00.933 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:00.485+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:31.909322+0000 front 2024-09-11T08:20:31.909100+0000 (oldest deadline 2024-09-11T08:20:56.009130+0000) 2024-09-11T08:21:00.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:00.676+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:21:00.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:00.676+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:32.604870+0000 front 2024-09-11T08:20:32.604797+0000 (oldest deadline 2024-09-11T08:20:56.104801+0000) 2024-09-11T08:21:01.433 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:21:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:21:00.959+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:23.477782+0000 front 2024-09-11T08:20:23.478003+0000 (oldest deadline 2024-09-11T08:20:46.377657+0000) 2024-09-11T08:21:01.434 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:21:00 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:21:00.959+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:28.678582+0000 front 2024-09-11T08:20:28.678543+0000 (oldest deadline 2024-09-11T08:20:54.578672+0000) 2024-09-11T08:21:01.931 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:01 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:01.483+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:21:01.931 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:01 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:01.483+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:31.909322+0000 front 2024-09-11T08:20:31.909100+0000 (oldest deadline 2024-09-11T08:20:56.009130+0000) 2024-09-11T08:21:01.932 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:01 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:01.650+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:21:01.932 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:01 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:01.650+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:32.604870+0000 front 2024-09-11T08:20:32.604797+0000 (oldest deadline 2024-09-11T08:20:56.104801+0000) 2024-09-11T08:21:01.932 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:21:01 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:21:01.931+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:23.477782+0000 front 2024-09-11T08:20:23.478003+0000 (oldest deadline 2024-09-11T08:20:46.377657+0000) 2024-09-11T08:21:01.932 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:21:01 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:21:01.931+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:28.678582+0000 front 2024-09-11T08:20:28.678543+0000 (oldest deadline 2024-09-11T08:20:54.578672+0000) 2024-09-11T08:21:01.932 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:21:01 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:21:01.931+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:39.780030+0000 front 2024-09-11T08:20:39.779998+0000 (oldest deadline 2024-09-11T08:21:01.479950+0000) 2024-09-11T08:21:02.893 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:02 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:02.670+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:21:02.893 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:02 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:02.670+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:32.604870+0000 front 2024-09-11T08:20:32.604797+0000 (oldest deadline 2024-09-11T08:20:56.104801+0000) 2024-09-11T08:21:02.893 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:02 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:02.500+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:21:02.893 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:02 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:02.500+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:31.909322+0000 front 2024-09-11T08:20:31.909100+0000 (oldest deadline 2024-09-11T08:20:56.009130+0000) 2024-09-11T08:21:02.893 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:02 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:02.500+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:36.009723+0000 front 2024-09-11T08:20:36.009708+0000 (oldest deadline 2024-09-11T08:21:01.909536+0000) 2024-09-11T08:21:03.183 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:21:02 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:21:02.892+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:23.477782+0000 front 2024-09-11T08:20:23.478003+0000 (oldest deadline 2024-09-11T08:20:46.377657+0000) 2024-09-11T08:21:03.183 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:21:02 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:21:02.892+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:28.678582+0000 front 2024-09-11T08:20:28.678543+0000 (oldest deadline 2024-09-11T08:20:54.578672+0000) 2024-09-11T08:21:03.183 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:21:02 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5[44840]: 2024-09-11T08:21:02.892+0000 7fc69bbd0640 -1 osd.5 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:39.780030+0000 front 2024-09-11T08:20:39.779998+0000 (oldest deadline 2024-09-11T08:21:01.479950+0000) 2024-09-11T08:21:03.895 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:03.575Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=2 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused" 2024-09-11T08:21:03.895 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:03.576Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused" 2024-09-11T08:21:03.896 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:03 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:03.576Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused" 2024-09-11T08:21:03.933 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:03 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:03.683+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:21:03.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:03 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:03.683+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:32.604870+0000 front 2024-09-11T08:20:32.604797+0000 (oldest deadline 2024-09-11T08:20:56.104801+0000) 2024-09-11T08:21:03.934 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:03 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:03.518+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:21:03.934 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:03 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:03.518+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:31.909322+0000 front 2024-09-11T08:20:31.909100+0000 (oldest deadline 2024-09-11T08:20:56.009130+0000) 2024-09-11T08:21:03.934 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:03 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:03.518+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:36.009723+0000 front 2024-09-11T08:20:36.009708+0000 (oldest deadline 2024-09-11T08:21:01.909536+0000) 2024-09-11T08:21:04.433 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:21:04 smithi146 podman[73065]: 2024-09-11 08:21:04.156278587 +0000 UTC m=+5.449049899 container died 7cb9aead02065129d4caf9ca1f38b90523d0fe050893be381e5e7d3801485f8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-11T08:21:04.774 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:21:04 smithi146 podman[73065]: 2024-09-11 08:21:04.539539215 +0000 UTC m=+5.832310529 container cleanup 7cb9aead02065129d4caf9ca1f38b90523d0fe050893be381e5e7d3801485f8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:21:04.774 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:21:04 smithi146 bash[73065]: ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5 2024-09-11T08:21:04.774 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:04.646+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:21:04.774 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:04.646+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:32.604870+0000 front 2024-09-11T08:20:32.604797+0000 (oldest deadline 2024-09-11T08:20:56.104801+0000) 2024-09-11T08:21:04.774 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:04.506+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:21:04.775 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:04.506+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:31.909322+0000 front 2024-09-11T08:20:31.909100+0000 (oldest deadline 2024-09-11T08:20:56.009130+0000) 2024-09-11T08:21:04.775 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:04 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:04.506+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:36.009723+0000 front 2024-09-11T08:20:36.009708+0000 (oldest deadline 2024-09-11T08:21:01.909536+0000) 2024-09-11T08:21:05.105 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:21:04 smithi146 podman[73086]: 2024-09-11 08:21:04.773998338 +0000 UTC m=+0.618865277 container remove 7cb9aead02065129d4caf9ca1f38b90523d0fe050893be381e5e7d3801485f8d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS) 2024-09-11T08:21:05.419 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:21:05 smithi146 podman[73180]: 2024-09-11 08:21:05.008283191 +0000 UTC m=+0.015983470 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:21:05.419 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:21:05 smithi146 podman[73180]: 2024-09-11 08:21:05.116477126 +0000 UTC m=+0.124177438 container create df28d1b8680668f917c4909be19693c7563d0b65b0706a4c27b2248709727809 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-11T08:21:05.419 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:21:05 smithi146 podman[73180]: 2024-09-11 08:21:05.335492306 +0000 UTC m=+0.343192572 container init df28d1b8680668f917c4909be19693c7563d0b65b0706a4c27b2248709727809 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-11T08:21:05.419 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:21:05 smithi146 podman[73180]: 2024-09-11 08:21:05.340149587 +0000 UTC m=+0.347849854 container start df28d1b8680668f917c4909be19693c7563d0b65b0706a4c27b2248709727809 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:21:05.419 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:21:05 smithi146 podman[73180]: 2024-09-11 08:21:05.418849402 +0000 UTC m=+0.426549676 container attach df28d1b8680668f917c4909be19693c7563d0b65b0706a4c27b2248709727809 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:21:05.681 INFO:journalctl@ceph.osd.5.smithi146.stdout:Sep 11 08:21:05 smithi146 podman[73180]: 2024-09-11 08:21:05.526587667 +0000 UTC m=+0.534287936 container died df28d1b8680668f917c4909be19693c7563d0b65b0706a4c27b2248709727809 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-11T08:21:05.681 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:05 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:05.502+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:21:05.681 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:05 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:05.502+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:31.909322+0000 front 2024-09-11T08:20:31.909100+0000 (oldest deadline 2024-09-11T08:20:56.009130+0000) 2024-09-11T08:21:05.681 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:05 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:05.502+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:36.009723+0000 front 2024-09-11T08:20:36.009708+0000 (oldest deadline 2024-09-11T08:21:01.909536+0000) 2024-09-11T08:21:05.933 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:05 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:05.680+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:21:05.933 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:05 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:05.680+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:32.604870+0000 front 2024-09-11T08:20:32.604797+0000 (oldest deadline 2024-09-11T08:20:56.104801+0000) 2024-09-11T08:21:06.145 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.5.service' 2024-09-11T08:21:06.183 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:21:06.184 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-11T08:21:06.184 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-11T08:21:06.184 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.6 2024-09-11T08:21:06.433 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:06 smithi146 systemd[1]: Stopping Ceph osd.6 for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:21:06.933 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:06 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:06.669+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:21:06.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:06 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:06.669+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:32.604870+0000 front 2024-09-11T08:20:32.604797+0000 (oldest deadline 2024-09-11T08:20:56.104801+0000) 2024-09-11T08:21:06.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:06 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:06.669+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:40.106187+0000 front 2024-09-11T08:20:40.106123+0000 (oldest deadline 2024-09-11T08:21:06.006119+0000) 2024-09-11T08:21:06.934 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:06 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:06.507+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:21:06.934 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:06 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:06.507+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:31.909322+0000 front 2024-09-11T08:20:31.909100+0000 (oldest deadline 2024-09-11T08:20:56.009130+0000) 2024-09-11T08:21:06.934 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:06 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:06.507+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:36.009723+0000 front 2024-09-11T08:20:36.009708+0000 (oldest deadline 2024-09-11T08:21:01.909536+0000) 2024-09-11T08:21:06.934 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:06 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:06.570+0000 7fcd181bd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:21:06.934 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:06 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:06.570+0000 7fcd181bd640 -1 osd.6 767 *** Got signal Terminated *** 2024-09-11T08:21:06.934 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:06 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:06.570+0000 7fcd181bd640 -1 osd.6 767 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:21:07.934 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:07 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:07.463+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:21:07.934 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:07 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:07.463+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:31.909322+0000 front 2024-09-11T08:20:31.909100+0000 (oldest deadline 2024-09-11T08:20:56.009130+0000) 2024-09-11T08:21:07.934 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:07 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:07.463+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:36.009723+0000 front 2024-09-11T08:20:36.009708+0000 (oldest deadline 2024-09-11T08:21:01.909536+0000) 2024-09-11T08:21:07.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:07 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:07.641+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:21:07.935 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:07 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:07.641+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:32.604870+0000 front 2024-09-11T08:20:32.604797+0000 (oldest deadline 2024-09-11T08:20:56.104801+0000) 2024-09-11T08:21:07.935 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:07 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:07.641+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:40.106187+0000 front 2024-09-11T08:20:40.106123+0000 (oldest deadline 2024-09-11T08:21:06.006119+0000) 2024-09-11T08:21:08.933 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:08.480+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:21:08.933 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:08.480+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:31.909322+0000 front 2024-09-11T08:20:31.909100+0000 (oldest deadline 2024-09-11T08:20:56.009130+0000) 2024-09-11T08:21:08.933 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:08.480+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:36.009723+0000 front 2024-09-11T08:20:36.009708+0000 (oldest deadline 2024-09-11T08:21:01.909536+0000) 2024-09-11T08:21:08.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:08.639+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:21:08.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:08.639+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:32.604870+0000 front 2024-09-11T08:20:32.604797+0000 (oldest deadline 2024-09-11T08:20:56.104801+0000) 2024-09-11T08:21:08.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:08 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:08.639+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:40.106187+0000 front 2024-09-11T08:20:40.106123+0000 (oldest deadline 2024-09-11T08:21:06.006119+0000) 2024-09-11T08:21:09.933 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:09 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:09.435+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:21:09.934 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:09 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:09.435+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:31.909322+0000 front 2024-09-11T08:20:31.909100+0000 (oldest deadline 2024-09-11T08:20:56.009130+0000) 2024-09-11T08:21:09.934 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:09 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:09.435+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:36.009723+0000 front 2024-09-11T08:20:36.009708+0000 (oldest deadline 2024-09-11T08:21:01.909536+0000) 2024-09-11T08:21:09.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:09 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:09.662+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:21:09.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:09 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:09.662+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:32.604870+0000 front 2024-09-11T08:20:32.604797+0000 (oldest deadline 2024-09-11T08:20:56.104801+0000) 2024-09-11T08:21:09.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:09 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:09.662+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:40.106187+0000 front 2024-09-11T08:20:40.106123+0000 (oldest deadline 2024-09-11T08:21:06.006119+0000) 2024-09-11T08:21:10.645 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:10 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:10.150Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused" 2024-09-11T08:21:10.645 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:10 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:10.151Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused" 2024-09-11T08:21:10.645 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:10 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:10.151Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused" 2024-09-11T08:21:10.933 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:10 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:10.483+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:21:10.934 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:10 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:10.483+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:31.909322+0000 front 2024-09-11T08:20:31.909100+0000 (oldest deadline 2024-09-11T08:20:56.009130+0000) 2024-09-11T08:21:10.934 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:10 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:10.483+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:36.009723+0000 front 2024-09-11T08:20:36.009708+0000 (oldest deadline 2024-09-11T08:21:01.909536+0000) 2024-09-11T08:21:10.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:10 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:10.637+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:21:10.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:10 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:10.637+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:32.604870+0000 front 2024-09-11T08:20:32.604797+0000 (oldest deadline 2024-09-11T08:20:56.104801+0000) 2024-09-11T08:21:10.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:10 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:10.637+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:40.106187+0000 front 2024-09-11T08:20:40.106123+0000 (oldest deadline 2024-09-11T08:21:06.006119+0000) 2024-09-11T08:21:10.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:10 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:10.637+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6815 osd.3 since back 2024-09-11T08:20:46.006657+0000 front 2024-09-11T08:20:46.006678+0000 (oldest deadline 2024-09-11T08:21:10.106521+0000) 2024-09-11T08:21:11.933 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:11.449+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:21.908232+0000 front 2024-09-11T08:20:21.908246+0000 (oldest deadline 2024-09-11T08:20:47.208138+0000) 2024-09-11T08:21:11.933 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:11.449+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:31.909322+0000 front 2024-09-11T08:20:31.909100+0000 (oldest deadline 2024-09-11T08:20:56.009130+0000) 2024-09-11T08:21:11.934 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6[51183]: 2024-09-11T08:21:11.449+0000 7fcd15fd8640 -1 osd.6 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:36.009723+0000 front 2024-09-11T08:20:36.009708+0000 (oldest deadline 2024-09-11T08:21:01.909536+0000) 2024-09-11T08:21:11.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:11.641+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:21:11.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:11.641+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:32.604870+0000 front 2024-09-11T08:20:32.604797+0000 (oldest deadline 2024-09-11T08:20:56.104801+0000) 2024-09-11T08:21:11.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:11.641+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:40.106187+0000 front 2024-09-11T08:20:40.106123+0000 (oldest deadline 2024-09-11T08:21:06.006119+0000) 2024-09-11T08:21:11.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:11 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:11.641+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6815 osd.3 since back 2024-09-11T08:20:46.006657+0000 front 2024-09-11T08:20:46.006678+0000 (oldest deadline 2024-09-11T08:21:10.106521+0000) 2024-09-11T08:21:12.368 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:12 smithi146 podman[73310]: 2024-09-11 08:21:12.09675457 +0000 UTC m=+5.692562900 container died f0d8f1fa0bd40b3e1d1a839cb9c241af0b8c1eb2f42dbb83dbde66acb74bc31d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-11T08:21:12.647 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:12 smithi146 podman[73310]: 2024-09-11 08:21:12.466454102 +0000 UTC m=+6.062262448 container cleanup f0d8f1fa0bd40b3e1d1a839cb9c241af0b8c1eb2f42dbb83dbde66acb74bc31d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-11T08:21:12.647 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:12 smithi146 bash[73310]: ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6 2024-09-11T08:21:12.936 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:12 smithi146 podman[73331]: 2024-09-11 08:21:12.734396923 +0000 UTC m=+0.635676485 container remove f0d8f1fa0bd40b3e1d1a839cb9c241af0b8c1eb2f42dbb83dbde66acb74bc31d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-11T08:21:12.937 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:12 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:12.647+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:21:12.937 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:12 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:12.647+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:32.604870+0000 front 2024-09-11T08:20:32.604797+0000 (oldest deadline 2024-09-11T08:20:56.104801+0000) 2024-09-11T08:21:12.937 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:12 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:12.647+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:40.106187+0000 front 2024-09-11T08:20:40.106123+0000 (oldest deadline 2024-09-11T08:21:06.006119+0000) 2024-09-11T08:21:12.937 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:12 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:12.647+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6815 osd.3 since back 2024-09-11T08:20:46.006657+0000 front 2024-09-11T08:20:46.006678+0000 (oldest deadline 2024-09-11T08:21:10.106521+0000) 2024-09-11T08:21:13.387 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:13 smithi146 podman[73426]: 2024-09-11 08:21:13.068529659 +0000 UTC m=+0.103035652 container create ede28a28cc8b757369c02c8c27b339b8e96d95f95c69cdbdff2386cd0f5dd826 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-11T08:21:13.388 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:13 smithi146 podman[73426]: 2024-09-11 08:21:12.98139798 +0000 UTC m=+0.015903975 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:21:13.388 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:13 smithi146 podman[73426]: 2024-09-11 08:21:13.27936561 +0000 UTC m=+0.313871605 container init ede28a28cc8b757369c02c8c27b339b8e96d95f95c69cdbdff2386cd0f5dd826 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:21:13.388 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:13 smithi146 podman[73426]: 2024-09-11 08:21:13.283912367 +0000 UTC m=+0.318418361 container start ede28a28cc8b757369c02c8c27b339b8e96d95f95c69cdbdff2386cd0f5dd826 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-11T08:21:13.388 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:13 smithi146 podman[73426]: 2024-09-11 08:21:13.38759615 +0000 UTC m=+0.422102147 container attach ede28a28cc8b757369c02c8c27b339b8e96d95f95c69cdbdff2386cd0f5dd826 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-11T08:21:13.657 INFO:journalctl@ceph.osd.6.smithi146.stdout:Sep 11 08:21:13 smithi146 podman[73426]: 2024-09-11 08:21:13.45906303 +0000 UTC m=+0.493569031 container died ede28a28cc8b757369c02c8c27b339b8e96d95f95c69cdbdff2386cd0f5dd826 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True) 2024-09-11T08:21:13.895 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:13 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:13.576Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=2 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused" 2024-09-11T08:21:13.895 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:13 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:13.576Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused" 2024-09-11T08:21:13.895 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:13 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:13.577Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused" 2024-09-11T08:21:13.933 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:13 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:13.656+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:21:13.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:13 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:13.656+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:32.604870+0000 front 2024-09-11T08:20:32.604797+0000 (oldest deadline 2024-09-11T08:20:56.104801+0000) 2024-09-11T08:21:13.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:13 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:13.656+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:40.106187+0000 front 2024-09-11T08:20:40.106123+0000 (oldest deadline 2024-09-11T08:21:06.006119+0000) 2024-09-11T08:21:13.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:13 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:13.656+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6815 osd.3 since back 2024-09-11T08:20:46.006657+0000 front 2024-09-11T08:20:46.006678+0000 (oldest deadline 2024-09-11T08:21:10.106521+0000) 2024-09-11T08:21:14.074 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.6.service' 2024-09-11T08:21:14.111 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:21:14.111 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-11T08:21:14.111 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-11T08:21:14.112 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.7 2024-09-11T08:21:14.430 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:14 smithi146 systemd[1]: Stopping Ceph osd.7 for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:21:14.683 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:14.428+0000 7f63143d9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:21:14.683 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:14.428+0000 7f63143d9640 -1 osd.7 767 *** Got signal Terminated *** 2024-09-11T08:21:14.683 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:14.428+0000 7f63143d9640 -1 osd.7 767 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T08:21:15.183 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:14.688+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:21:15.184 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:14.688+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:32.604870+0000 front 2024-09-11T08:20:32.604797+0000 (oldest deadline 2024-09-11T08:20:56.104801+0000) 2024-09-11T08:21:15.184 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:14.688+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:40.106187+0000 front 2024-09-11T08:20:40.106123+0000 (oldest deadline 2024-09-11T08:21:06.006119+0000) 2024-09-11T08:21:15.184 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:14 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:14.688+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6815 osd.3 since back 2024-09-11T08:20:46.006657+0000 front 2024-09-11T08:20:46.006678+0000 (oldest deadline 2024-09-11T08:21:10.106521+0000) 2024-09-11T08:21:15.933 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:15 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:15.680+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:21:15.933 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:15 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:15.680+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:32.604870+0000 front 2024-09-11T08:20:32.604797+0000 (oldest deadline 2024-09-11T08:20:56.104801+0000) 2024-09-11T08:21:15.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:15 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:15.680+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:40.106187+0000 front 2024-09-11T08:20:40.106123+0000 (oldest deadline 2024-09-11T08:21:06.006119+0000) 2024-09-11T08:21:15.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:15 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:15.680+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6815 osd.3 since back 2024-09-11T08:20:46.006657+0000 front 2024-09-11T08:20:46.006678+0000 (oldest deadline 2024-09-11T08:21:10.106521+0000) 2024-09-11T08:21:17.183 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:16 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:16.719+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:21:17.184 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:16 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:16.719+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:32.604870+0000 front 2024-09-11T08:20:32.604797+0000 (oldest deadline 2024-09-11T08:20:56.104801+0000) 2024-09-11T08:21:17.184 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:16 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:16.719+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:40.106187+0000 front 2024-09-11T08:20:40.106123+0000 (oldest deadline 2024-09-11T08:21:06.006119+0000) 2024-09-11T08:21:17.184 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:16 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:16.719+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6815 osd.3 since back 2024-09-11T08:20:46.006657+0000 front 2024-09-11T08:20:46.006678+0000 (oldest deadline 2024-09-11T08:21:10.106521+0000) 2024-09-11T08:21:18.085 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:17 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:17.702+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:21:18.085 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:17 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:17.702+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:32.604870+0000 front 2024-09-11T08:20:32.604797+0000 (oldest deadline 2024-09-11T08:20:56.104801+0000) 2024-09-11T08:21:18.085 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:17 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:17.702+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:40.106187+0000 front 2024-09-11T08:20:40.106123+0000 (oldest deadline 2024-09-11T08:21:06.006119+0000) 2024-09-11T08:21:18.085 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:17 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:17.702+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6815 osd.3 since back 2024-09-11T08:20:46.006657+0000 front 2024-09-11T08:20:46.006678+0000 (oldest deadline 2024-09-11T08:21:10.106521+0000) 2024-09-11T08:21:18.086 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:17 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:17.702+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.146:6802 osd.4 since back 2024-09-11T08:20:51.807379+0000 front 2024-09-11T08:20:51.807301+0000 (oldest deadline 2024-09-11T08:21:17.107374+0000) 2024-09-11T08:21:18.933 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:18 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:18.672+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6803 osd.0 since back 2024-09-11T08:20:22.703558+0000 front 2024-09-11T08:20:22.703673+0000 (oldest deadline 2024-09-11T08:20:49.703828+0000) 2024-09-11T08:21:18.933 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:18 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:18.672+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6807 osd.1 since back 2024-09-11T08:20:32.604870+0000 front 2024-09-11T08:20:32.604797+0000 (oldest deadline 2024-09-11T08:20:56.104801+0000) 2024-09-11T08:21:18.933 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:18 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:18.672+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6811 osd.2 since back 2024-09-11T08:20:40.106187+0000 front 2024-09-11T08:20:40.106123+0000 (oldest deadline 2024-09-11T08:21:06.006119+0000) 2024-09-11T08:21:18.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:18 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:18.672+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.102:6815 osd.3 since back 2024-09-11T08:20:46.006657+0000 front 2024-09-11T08:20:46.006678+0000 (oldest deadline 2024-09-11T08:21:10.106521+0000) 2024-09-11T08:21:18.934 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:18 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7[57199]: 2024-09-11T08:21:18.672+0000 7f63121f4640 -1 osd.7 767 heartbeat_check: no reply from 172.21.15.146:6802 osd.4 since back 2024-09-11T08:20:51.807379+0000 front 2024-09-11T08:20:51.807301+0000 (oldest deadline 2024-09-11T08:21:17.107374+0000) 2024-09-11T08:21:20.169 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:19 smithi146 podman[73554]: 2024-09-11 08:21:19.914252073 +0000 UTC m=+5.577302960 container died abbc25857ea6159f4674a1e4f8b4283bda632dddba3499243d07d752e0a222d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-11T08:21:20.433 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:20 smithi146 podman[73554]: 2024-09-11 08:21:20.259754908 +0000 UTC m=+5.922805829 container cleanup abbc25857ea6159f4674a1e4f8b4283bda632dddba3499243d07d752e0a222d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-11T08:21:20.434 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:20 smithi146 bash[73554]: ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7 2024-09-11T08:21:20.645 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:20 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:20.151Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused" 2024-09-11T08:21:20.645 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:20 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:20.151Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused" 2024-09-11T08:21:20.645 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:20 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:20.152Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused" 2024-09-11T08:21:20.872 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:20 smithi146 podman[73586]: 2024-09-11 08:21:20.535639546 +0000 UTC m=+0.625273286 container remove abbc25857ea6159f4674a1e4f8b4283bda632dddba3499243d07d752e0a222d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7, GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T08:21:21.128 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:20 smithi146 podman[73678]: 2024-09-11 08:21:20.775388027 +0000 UTC m=+0.015771172 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T08:21:21.128 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:20 smithi146 podman[73678]: 2024-09-11 08:21:20.895067888 +0000 UTC m=+0.135451059 container create ace8b8a89df44696d816ba0ab9be89fb62c2bf5c0273c30b1d10eeff598cd0c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T08:21:21.128 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:21 smithi146 podman[73678]: 2024-09-11 08:21:21.121922027 +0000 UTC m=+0.362305159 container init ace8b8a89df44696d816ba0ab9be89fb62c2bf5c0273c30b1d10eeff598cd0c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-11T08:21:21.128 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:21 smithi146 podman[73678]: 2024-09-11 08:21:21.127186012 +0000 UTC m=+0.367569149 container start ace8b8a89df44696d816ba0ab9be89fb62c2bf5c0273c30b1d10eeff598cd0c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-11T08:21:21.433 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:21 smithi146 podman[73678]: 2024-09-11 08:21:21.247245646 +0000 UTC m=+0.487628784 container attach ace8b8a89df44696d816ba0ab9be89fb62c2bf5c0273c30b1d10eeff598cd0c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-11T08:21:21.434 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 11 08:21:21 smithi146 podman[73678]: 2024-09-11 08:21:21.300595617 +0000 UTC m=+0.540978760 container died ace8b8a89df44696d816ba0ab9be89fb62c2bf5c0273c30b1d10eeff598cd0c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-11T08:21:21.873 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@osd.7.service' 2024-09-11T08:21:21.913 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:21:21.914 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-11T08:21:21.914 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-11T08:21:21.914 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-2051892c-7013-11ef-bcea-c7b262605968@rgw.foo.a 2024-09-11T08:21:22.210 INFO:journalctl@ceph.rgw.foo.a.smithi102.stdout:Sep 11 08:21:21 smithi102 systemd[1]: Stopping Ceph rgw.foo.a for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:21:22.645 INFO:journalctl@ceph.rgw.foo.a.smithi102.stdout:Sep 11 08:21:22 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-rgw-foo-a[69534]: 2024-09-11T08:21:22.209+0000 7fa9f62e8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T08:21:22.645 INFO:journalctl@ceph.rgw.foo.a.smithi102.stdout:Sep 11 08:21:22 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-rgw-foo-a[69534]: 2024-09-11T08:21:22.209+0000 7fa9f9fef7c0 -1 shutting down 2024-09-11T08:21:23.895 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:23 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:23.577Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=2 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused" 2024-09-11T08:21:23.895 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:23 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:23.578Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused" 2024-09-11T08:21:23.895 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:23 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:23.578Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused" 2024-09-11T08:21:30.469 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:30 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:30.152Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused" 2024-09-11T08:21:30.469 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:30 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:30.153Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused" 2024-09-11T08:21:30.469 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:30 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:30.153Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused" 2024-09-11T08:21:31.895 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:31 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:31.625Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused" 2024-09-11T08:21:31.895 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:31 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:31.625Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused" 2024-09-11T08:21:32.483 INFO:journalctl@ceph.rgw.foo.a.smithi102.stdout:Sep 11 08:21:32 smithi102 bash[151891]: time="2024-09-11T08:21:32Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-2051892c-7013-11ef-bcea-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-11T08:21:32.483 INFO:journalctl@ceph.rgw.foo.a.smithi102.stdout:Sep 11 08:21:32 smithi102 podman[151891]: 2024-09-11 08:21:32.231055537 +0000 UTC m=+10.120885212 container died 58d5114f6458a81816dd1f418e5796a55dbd7776e9029633c300606a39dfab3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-rgw-foo-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T08:21:32.757 INFO:journalctl@ceph.rgw.foo.a.smithi102.stdout:Sep 11 08:21:32 smithi102 podman[151891]: 2024-09-11 08:21:32.574366355 +0000 UTC m=+10.464196023 container cleanup 58d5114f6458a81816dd1f418e5796a55dbd7776e9029633c300606a39dfab3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-rgw-foo-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-11T08:21:32.757 INFO:journalctl@ceph.rgw.foo.a.smithi102.stdout:Sep 11 08:21:32 smithi102 bash[151891]: ceph-2051892c-7013-11ef-bcea-c7b262605968-rgw-foo-a 2024-09-11T08:21:32.998 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@rgw.foo.a.service' 2024-09-11T08:21:33.020 INFO:journalctl@ceph.rgw.foo.a.smithi102.stdout:Sep 11 08:21:32 smithi102 podman[151928]: 2024-09-11 08:21:32.757547489 +0000 UTC m=+0.526903116 container remove 58d5114f6458a81816dd1f418e5796a55dbd7776e9029633c300606a39dfab3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-rgw-foo-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-11T08:21:33.021 INFO:journalctl@ceph.rgw.foo.a.smithi102.stdout:Sep 11 08:21:32 smithi102 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@rgw.foo.a.service: Main process exited, code=exited, status=137/n/a 2024-09-11T08:21:33.021 INFO:journalctl@ceph.rgw.foo.a.smithi102.stdout:Sep 11 08:21:32 smithi102 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@rgw.foo.a.service: Failed with result 'exit-code'. 2024-09-11T08:21:33.021 INFO:journalctl@ceph.rgw.foo.a.smithi102.stdout:Sep 11 08:21:32 smithi102 systemd[1]: Stopped Ceph rgw.foo.a for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T08:21:33.021 INFO:journalctl@ceph.rgw.foo.a.smithi102.stdout:Sep 11 08:21:32 smithi102 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@rgw.foo.a.service: Consumed 3.640s CPU time. 2024-09-11T08:21:33.034 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:21:33.035 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-11T08:21:33.035 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-11T08:21:33.035 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-2051892c-7013-11ef-bcea-c7b262605968@prometheus.a 2024-09-11T08:21:33.340 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:21:33 smithi146 systemd[1]: Stopping Ceph prometheus.a for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:21:33.639 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:21:33 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:21:33.340Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:21:33.640 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:21:33 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:21:33.340Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-11T08:21:33.640 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:21:33 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:21:33.340Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-11T08:21:33.640 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:21:33 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:21:33.340Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-11T08:21:33.640 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:21:33 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:21:33.340Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-11T08:21:33.640 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:21:33 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:21:33.340Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-11T08:21:33.640 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:21:33 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:21:33.340Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-11T08:21:33.640 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:21:33 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:21:33.340Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-11T08:21:33.640 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:21:33 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:21:33.340Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-11T08:21:33.640 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:21:33 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:21:33.341Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-11T08:21:33.640 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:21:33 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:21:33.341Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-11T08:21:33.640 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:21:33 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a[68057]: ts=2024-09-11T08:21:33.341Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-11T08:21:33.640 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:21:33 smithi146 podman[73805]: 2024-09-11 08:21:33.350911824 +0000 UTC m=+0.119221243 container died 83beeb4391f4ce1f3c5137ce595b8cb93ab5910ba55c192dee2f7488f170346c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:21:33.895 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:33 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:33.577Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=2 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused" 2024-09-11T08:21:33.895 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:33 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:33.578Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi102.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.102:8443: connect: connection refused" 2024-09-11T08:21:33.895 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:33 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:33.578Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi146.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.146:8443: connect: connection refused" 2024-09-11T08:21:33.933 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:21:33 smithi146 podman[73805]: 2024-09-11 08:21:33.746202798 +0000 UTC m=+0.514512262 container cleanup 83beeb4391f4ce1f3c5137ce595b8cb93ab5910ba55c192dee2f7488f170346c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T08:21:33.933 INFO:journalctl@ceph.prometheus.a.smithi146.stdout:Sep 11 08:21:33 smithi146 bash[73805]: ceph-2051892c-7013-11ef-bcea-c7b262605968-prometheus-a 2024-09-11T08:21:34.117 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-2051892c-7013-11ef-bcea-c7b262605968@prometheus.a.service' 2024-09-11T08:21:34.152 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T08:21:34.153 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-11T08:21:34.153 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm rm-cluster --fsid 2051892c-7013-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T08:21:36.086 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:35 smithi102 systemd[1]: Stopping Ceph alertmanager.a for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:21:36.086 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:36 smithi102 ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a[78959]: ts=2024-09-11T08:21:36.055Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-11T08:21:36.086 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:36 smithi102 podman[152360]: 2024-09-11 08:21:36.086304729 +0000 UTC m=+0.135396540 container died 5068c22cecbc535e57a2e446c714409f1d0fc43c453113db057a942ad6a16810 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:21:36.612 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:36 smithi102 podman[152360]: 2024-09-11 08:21:36.419989868 +0000 UTC m=+0.469081684 container cleanup 5068c22cecbc535e57a2e446c714409f1d0fc43c453113db057a942ad6a16810 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:21:36.613 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:36 smithi102 bash[152360]: ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a 2024-09-11T08:21:36.895 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:36 smithi102 podman[152376]: 2024-09-11 08:21:36.612882763 +0000 UTC m=+0.521322087 container remove 5068c22cecbc535e57a2e446c714409f1d0fc43c453113db057a942ad6a16810 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T08:21:36.895 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:36 smithi102 podman[152376]: 2024-09-11 08:21:36.688630465 +0000 UTC m=+0.597069789 volume remove a46821dc73bc5431bd08c28065be24a775a6537b9b830e8281916cce7619b7fd 2024-09-11T08:21:37.395 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:21:37 smithi102 systemd[1]: Stopping Ceph node-exporter.a for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:21:37.395 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:36 smithi102 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-11T08:21:37.396 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:36 smithi102 systemd[1]: Stopped Ceph alertmanager.a for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T08:21:37.396 INFO:journalctl@ceph.alertmanager.a.smithi102.stdout:Sep 11 08:21:36 smithi102 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@alertmanager.a.service: Consumed 1.811s CPU time. 2024-09-11T08:21:37.692 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:21:37 smithi102 podman[152491]: 2024-09-11 08:21:37.437623372 +0000 UTC m=+0.126194433 container died 6f44a611aab2970264a5b67d2745838a8864cc21baa79d9e1d0102e1bf8c3109 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:21:38.058 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:21:37 smithi102 podman[152491]: 2024-09-11 08:21:37.791358959 +0000 UTC m=+0.479930061 container cleanup 6f44a611aab2970264a5b67d2745838a8864cc21baa79d9e1d0102e1bf8c3109 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:21:38.059 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:21:37 smithi102 bash[152491]: ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a 2024-09-11T08:21:38.345 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:21:38 smithi102 podman[152511]: 2024-09-11 08:21:38.059013966 +0000 UTC m=+0.625391447 container remove 6f44a611aab2970264a5b67d2745838a8864cc21baa79d9e1d0102e1bf8c3109 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T08:21:38.345 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:21:38 smithi102 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-11T08:21:38.345 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:21:38 smithi102 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-11T08:21:38.345 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:21:38 smithi102 systemd[1]: Stopped Ceph node-exporter.a for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T08:21:38.345 INFO:journalctl@ceph.node-exporter.a.smithi102.stdout:Sep 11 08:21:38 smithi102 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@node-exporter.a.service: Consumed 3.532s CPU time. 2024-09-11T08:21:38.822 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm rm-cluster --fsid 2051892c-7013-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T08:21:40.433 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:21:40 smithi146 systemd[1]: Stopping Ceph iscsi.iscsi.a for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:21:40.433 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:21:40 smithi146 ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a[62763]: debug Shutdown received 2024-09-11T08:21:50.684 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:21:50 smithi146 bash[74191]: time="2024-09-11T08:21:50Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-11T08:21:50.684 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:21:50 smithi146 podman[74191]: 2024-09-11 08:21:50.406294921 +0000 UTC m=+10.102156646 container died b1dab48cbbb6fcc4cb24f1ad9c901746782d38974d9c0a21d4e0e4445b9c18b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-11T08:21:51.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:21:50 smithi146 podman[74191]: 2024-09-11 08:21:50.7616512 +0000 UTC m=+10.457512922 container cleanup b1dab48cbbb6fcc4cb24f1ad9c901746782d38974d9c0a21d4e0e4445b9c18b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-11T08:21:51.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:21:50 smithi146 bash[74191]: ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a 2024-09-11T08:21:51.278 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:21:51 smithi146 podman[74214]: 2024-09-11 08:21:51.012817338 +0000 UTC m=+0.604111852 container remove b1dab48cbbb6fcc4cb24f1ad9c901746782d38974d9c0a21d4e0e4445b9c18b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-11T08:21:51.278 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:21:51 smithi146 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-11T08:21:51.533 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:21:51 smithi146 podman[74260]: 2024-09-11 08:21:51.277893496 +0000 UTC m=+0.105890451 container died 810393cb3f2c802c90e3ccca04d945f729f8bdd99467fb28929375f5408c493d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-11T08:21:51.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:21:51 smithi146 podman[74260]: 2024-09-11 08:21:51.632915261 +0000 UTC m=+0.460912258 container cleanup 810393cb3f2c802c90e3ccca04d945f729f8bdd99467fb28929375f5408c493d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T08:21:51.852 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:21:51 smithi146 bash[74260]: ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-11T08:21:51.853 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:21:51 smithi146 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-11T08:21:51.853 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:21:51 smithi146 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 62591 (conmon) remains running after unit stopped. 2024-09-11T08:21:51.853 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:21:51 smithi146 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 74277 (podman) remains running after unit stopped. 2024-09-11T08:21:51.853 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:21:51 smithi146 systemd[1]: Stopped Ceph iscsi.iscsi.a for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T08:21:51.853 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:21:51 smithi146 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Consumed 3.243s CPU time. 2024-09-11T08:21:52.183 INFO:journalctl@ceph.iscsi.iscsi.a.smithi146.stdout:Sep 11 08:21:51 smithi146 podman[74277]: 2024-09-11 08:21:51.93477423 +0000 UTC m=+0.656887607 container remove 810393cb3f2c802c90e3ccca04d945f729f8bdd99467fb28929375f5408c493d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-11T08:21:52.184 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:21:52 smithi146 systemd[1]: Stopping Ceph grafana.a for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:21:52.621 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:21:52 smithi146 podman[74400]: 2024-09-11 08:21:52.31720456 +0000 UTC m=+0.109823319 container died 1dd8b360aa501f9edd30fd0b7f0894060e546f3e7054a6af2b9b4b9e368a8d44 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a, vendor=Red Hat, Inc., version=8.5, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, summary=Grafana Container configured for Ceph mgr/dashboard integration, build-date=2022-04-29T11:48:11.055897, name=ubi8, description=Ceph Grafana Container, io.openshift.expose-services=, release=239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, distribution-scope=public, vcs-type=git, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8, architecture=x86_64, com.redhat.component=ubi8-container, io.buildah.version=1.32.0, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, maintainer=Paul Cuzner , com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.) 2024-09-11T08:21:52.621 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:21:52 smithi146 podman[74400]: 2024-09-11 08:21:52.62062143 +0000 UTC m=+0.413240188 container cleanup 1dd8b360aa501f9edd30fd0b7f0894060e546f3e7054a6af2b9b4b9e368a8d44 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., architecture=x86_64, summary=Grafana Container configured for Ceph mgr/dashboard integration, maintainer=Paul Cuzner , vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.component=ubi8-container, name=ubi8, vendor=Red Hat, Inc., version=8.5, io.openshift.expose-services=, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-type=git, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, build-date=2022-04-29T11:48:11.055897, io.buildah.version=1.32.0, io.openshift.tags=base rhel8, distribution-scope=public, description=Ceph Grafana Container, release=239.1651231664) 2024-09-11T08:21:52.933 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:21:52 smithi146 bash[74400]: ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a 2024-09-11T08:21:52.933 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:21:52 smithi146 podman[74418]: 2024-09-11 08:21:52.834706486 +0000 UTC m=+0.518702208 container remove 1dd8b360aa501f9edd30fd0b7f0894060e546f3e7054a6af2b9b4b9e368a8d44 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-grafana-a, build-date=2022-04-29T11:48:11.055897, release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.expose-services=, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, description=Ceph Grafana Container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.component=ubi8-container, io.buildah.version=1.32.0, maintainer=Paul Cuzner , version=8.5, vcs-type=git, vendor=Red Hat, Inc., distribution-scope=public, io.openshift.tags=base rhel8, architecture=x86_64, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.display-name=Red Hat Universal Base Image 8) 2024-09-11T08:21:52.933 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:21:52 smithi146 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-11T08:21:53.345 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:21:53 smithi146 systemd[1]: Stopping Ceph node-exporter.b for 2051892c-7013-11ef-bcea-c7b262605968... 2024-09-11T08:21:53.346 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:21:52 smithi146 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-11T08:21:53.346 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:21:52 smithi146 systemd[1]: Stopped Ceph grafana.a for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T08:21:53.346 INFO:journalctl@ceph.grafana.a.smithi146.stdout:Sep 11 08:21:52 smithi146 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@grafana.a.service: Consumed 11.318s CPU time. 2024-09-11T08:21:53.683 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:21:53 smithi146 podman[74541]: 2024-09-11 08:21:53.467449726 +0000 UTC m=+0.116481941 container died e4a63719b0a9b4b8dd964943bd319e32438794cfc50449c4e82b78945dc34257 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:21:53.972 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:21:53 smithi146 podman[74541]: 2024-09-11 08:21:53.762487022 +0000 UTC m=+0.411519238 container cleanup e4a63719b0a9b4b8dd964943bd319e32438794cfc50449c4e82b78945dc34257 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:21:53.972 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:21:53 smithi146 bash[74541]: ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b 2024-09-11T08:21:54.252 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:21:53 smithi146 podman[74560]: 2024-09-11 08:21:53.972074482 +0000 UTC m=+0.504117567 container remove e4a63719b0a9b4b8dd964943bd319e32438794cfc50449c4e82b78945dc34257 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-2051892c-7013-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T08:21:54.252 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:21:53 smithi146 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-11T08:21:54.252 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:21:54 smithi146 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-11T08:21:54.252 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:21:54 smithi146 systemd[1]: Stopped Ceph node-exporter.b for 2051892c-7013-11ef-bcea-c7b262605968. 2024-09-11T08:21:54.253 INFO:journalctl@ceph.node-exporter.b.smithi146.stdout:Sep 11 08:21:54 smithi146 systemd[1]: ceph-2051892c-7013-11ef-bcea-c7b262605968@node-exporter.b.service: Consumed 3.694s CPU time. 2024-09-11T08:21:54.748 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:21:54.782 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T08:21:54.809 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-11T08:21:54.811 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901242/remote/smithi102/crash 2024-09-11T08:21:54.812 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/crash -- . 2024-09-11T08:21:54.886 INFO:teuthology.orchestra.run.smithi102.stderr:tar: /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T08:21:54.886 INFO:teuthology.orchestra.run.smithi102.stderr:tar: Error is not recoverable: exiting now 2024-09-11T08:21:54.888 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901242/remote/smithi146/crash 2024-09-11T08:21:54.889 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/crash -- . 2024-09-11T08:21:54.959 INFO:teuthology.orchestra.run.smithi146.stderr:tar: /var/lib/ceph/2051892c-7013-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T08:21:54.959 INFO:teuthology.orchestra.run.smithi146.stderr:tar: Error is not recoverable: exiting now 2024-09-11T08:21:54.960 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-11T08:21:54.960 DEBUG:teuthology.orchestra.run.smithi102:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-09-11T08:21:54.999 INFO:teuthology.orchestra.run.smithi102.stderr:grep: /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph.log: No such file or directory 2024-09-11T08:21:55.001 INFO:tasks.cephadm:Compressing logs... 2024-09-11T08:21:55.002 DEBUG:teuthology.orchestra.run.smithi102:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T08:21:55.045 DEBUG:teuthology.orchestra.run.smithi146:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T08:21:55.071 INFO:teuthology.orchestra.run.smithi146.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/tcmu-runner.log 2024-09-11T08:21:55.072 INFO:teuthology.orchestra.run.smithi146.stderr:: No such file or directory 2024-09-11T08:21:55.072 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-11T08:21:55.072 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/tcmu-runner.log: gzip -5 --verbose -- /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T08:21:55.072 INFO:teuthology.orchestra.run.smithi146.stderr: 63.1% -- replaced with /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/tcmu-runner.log.gz 2024-09-11T08:21:55.072 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-11T08:21:55.072 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-11T08:21:55.072 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-11T08:21:55.072 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-11T08:21:55.072 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-11T08:21:55.074 INFO:teuthology.orchestra.run.smithi102.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-11T08:21:55.074 INFO:teuthology.orchestra.run.smithi102.stderr:: No such file or directory 2024-09-11T08:21:55.074 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-11T08:21:55.074 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T08:21:55.074 INFO:teuthology.orchestra.run.smithi102.stderr:gzip/var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.1.log: -5 --verbose -- /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-11T08:21:55.075 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-11T08:21:55.075 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-11T08:21:55.075 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-11T08:21:55.075 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T08:21:55.075 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T08:21:55.080 INFO:teuthology.orchestra.run.smithi146.stderr: 90.7% -- replaced with /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T08:21:55.080 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.2.log: /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-mgr.y.log: 90.9% -- replaced with /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T08:21:55.081 INFO:teuthology.orchestra.run.smithi146.stderr: 91.8% -- replaced with /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-11T08:21:55.084 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-11T08:21:55.084 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-11T08:21:55.088 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T08:21:55.115 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T08:21:55.125 INFO:teuthology.orchestra.run.smithi102.stderr: 93.2% -- replaced with /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-11T08:21:55.469 INFO:teuthology.orchestra.run.smithi102.stderr: 90.2% -- replaced with /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-11T08:21:56.221 INFO:teuthology.orchestra.run.smithi146.stderr: 92.4% -- replaced with /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-11T08:21:56.662 INFO:teuthology.orchestra.run.smithi102.stderr: 92.1% -- replaced with /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-11T08:21:57.320 INFO:teuthology.orchestra.run.smithi102.stderr: 91.7% -- replaced with /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-11T08:22:00.408 INFO:teuthology.orchestra.run.smithi146.stderr: 93.9% -- replaced with /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-11T08:22:00.685 INFO:teuthology.orchestra.run.smithi146.stderr: 93.9% -- replaced with /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-11T08:22:00.687 INFO:teuthology.orchestra.run.smithi146.stderr: 94.0% -- replaced with /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-11T08:22:00.847 INFO:teuthology.orchestra.run.smithi102.stderr: 94.0% -- replaced with /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-11T08:22:01.201 INFO:teuthology.orchestra.run.smithi102.stderr: 94.0% -- replaced with /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-11T08:22:01.257 INFO:teuthology.orchestra.run.smithi146.stderr: 94.0% -- replaced with /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-11T08:22:01.259 INFO:teuthology.orchestra.run.smithi146.stderr: 2024-09-11T08:22:01.259 INFO:teuthology.orchestra.run.smithi146.stderr:real 0m6.199s 2024-09-11T08:22:01.259 INFO:teuthology.orchestra.run.smithi146.stderr:user 0m23.075s 2024-09-11T08:22:01.259 INFO:teuthology.orchestra.run.smithi146.stderr:sys 0m0.830s 2024-09-11T08:22:01.292 INFO:teuthology.orchestra.run.smithi102.stderr: 94.1% -- replaced with /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-11T08:22:01.520 INFO:teuthology.orchestra.run.smithi102.stderr: 94.0% -- replaced with /var/log/ceph/2051892c-7013-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-11T08:22:01.522 INFO:teuthology.orchestra.run.smithi102.stderr: 2024-09-11T08:22:01.522 INFO:teuthology.orchestra.run.smithi102.stderr:real 0m6.463s 2024-09-11T08:22:01.522 INFO:teuthology.orchestra.run.smithi102.stderr:user 0m27.859s 2024-09-11T08:22:01.522 INFO:teuthology.orchestra.run.smithi102.stderr:sys 0m0.958s 2024-09-11T08:22:01.523 INFO:tasks.cephadm:Archiving logs... 2024-09-11T08:22:01.523 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901242/remote/smithi102/log 2024-09-11T08:22:01.524 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T08:22:05.273 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901242/remote/smithi146/log 2024-09-11T08:22:05.273 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T08:22:08.535 INFO:tasks.cephadm:Removing cluster... 2024-09-11T08:22:08.535 DEBUG:teuthology.orchestra.run.smithi102:> sudo cephadm rm-cluster --fsid 2051892c-7013-11ef-bcea-c7b262605968 --force 2024-09-11T08:22:09.036 DEBUG:teuthology.orchestra.run.smithi146:> sudo cephadm rm-cluster --fsid 2051892c-7013-11ef-bcea-c7b262605968 --force 2024-09-11T08:22:09.948 INFO:tasks.cephadm:Teardown complete 2024-09-11T08:22:09.949 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T08:22:09.959 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T08:22:09.959 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T08:22:09.962 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T08:22:10.034 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T08:22:10.136 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T08:22:10.136 DEBUG:teuthology.orchestra.run.smithi102:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:22:10.139 DEBUG:teuthology.orchestra.run.smithi146:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T08:22:10.153 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:22:10.153 INFO:teuthology.orchestra.run.smithi146.stderr:bash: line 1: ntpq: command not found 2024-09-11T08:22:10.159 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:22:10.159 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-09-11T08:22:10.159 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 21 -1398us[-1380us] +/- 49ms 2024-09-11T08:22:10.159 INFO:teuthology.orchestra.run.smithi102.stdout:^* hv02.front.sepia.ceph.com 3 6 377 21 +1105us[+1123us] +/- 56ms 2024-09-11T08:22:10.159 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 +590us[ +590us] +/- 70ms 2024-09-11T08:22:10.159 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:22:10.159 INFO:teuthology.orchestra.run.smithi146.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T08:22:10.159 INFO:teuthology.orchestra.run.smithi146.stdout:=============================================================================== 2024-09-11T08:22:10.159 INFO:teuthology.orchestra.run.smithi146.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 21 -1406us[-1416us] +/- 49ms 2024-09-11T08:22:10.160 INFO:teuthology.orchestra.run.smithi146.stdout:^* hv02.front.sepia.ceph.com 3 6 377 17 +1075us[+1066us] +/- 56ms 2024-09-11T08:22:10.160 INFO:teuthology.orchestra.run.smithi146.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 +550us[ +541us] +/- 70ms 2024-09-11T08:22:10.160 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T08:22:10.160 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T08:22:10.185 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T08:22:10.185 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T08:22:10.203 DEBUG:teuthology.orchestra.run.smithi102:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T08:22:10.284 DEBUG:teuthology.orchestra.run.smithi146:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T08:22:10.356 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T08:22:10.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:22:10.386 DEBUG:teuthology.orchestra.run.smithi146:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T08:22:10.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:22:10.414 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T08:22:10.424 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T08:22:10.437 INFO:teuthology.task.internal:Duration was 1999.183058 seconds 2024-09-11T08:22:10.438 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T08:22:10.502 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T08:22:10.502 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T08:22:10.505 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T08:22:10.543 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:22:10.543 INFO:teuthology.orchestra.run.smithi146.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T08:22:10.934 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T08:22:10.934 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi102.front.sepia.ceph.com 2024-09-11T08:22:10.935 DEBUG:teuthology.orchestra.run.smithi102:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T08:22:10.996 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi146.front.sepia.ceph.com 2024-09-11T08:22:10.996 DEBUG:teuthology.orchestra.run.smithi146:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T08:22:11.027 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T08:22:11.027 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T08:22:11.038 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T08:22:11.093 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T08:22:11.094 DEBUG:teuthology.orchestra.run.smithi102:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T08:22:11.105 DEBUG:teuthology.orchestra.run.smithi146:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T08:22:11.521 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T08:22:11.531 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T08:22:11.531 DEBUG:teuthology.orchestra.run.smithi102:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T08:22:11.556 DEBUG:teuthology.orchestra.run.smithi146:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T08:22:11.581 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T08:22:11.599 DEBUG:teuthology.orchestra.run.smithi102:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:22:11.601 DEBUG:teuthology.orchestra.run.smithi146:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:22:11.622 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = core 2024-09-11T08:22:11.644 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = core 2024-09-11T08:22:11.661 DEBUG:teuthology.orchestra.run.smithi102:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:22:11.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:22:11.689 DEBUG:teuthology.orchestra.run.smithi146:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T08:22:11.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T08:22:11.717 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T08:22:11.726 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T08:22:11.727 DEBUG:teuthology.misc:Transferring archived files from smithi102:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901242/remote/smithi102 2024-09-11T08:22:11.727 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T08:22:11.771 DEBUG:teuthology.misc:Transferring archived files from smithi146:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901242/remote/smithi146 2024-09-11T08:22:11.771 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T08:22:11.807 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T08:22:11.808 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T08:22:11.810 DEBUG:teuthology.orchestra.run.smithi146:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T08:22:11.857 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T08:22:11.868 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T08:22:11.868 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T08:22:11.894 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T08:22:11.894 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T08:22:11.896 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T08:22:11.909 INFO:teuthology.orchestra.run.smithi102.stdout: 265836 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 08:22 /home/ubuntu/cephtest 2024-09-11T08:22:11.911 INFO:teuthology.orchestra.run.smithi146.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 08:22 /home/ubuntu/cephtest 2024-09-11T08:22:11.912 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T08:22:11.937 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T08:22:12.000 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} duration: 1999.1830582618713 flavor: default owner: scheduled_gabrioux@teuthology success: true 2024-09-11T08:22:12.001 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T08:22:12.085 INFO:teuthology.run:pass